Optical id sensing using illumination light sources positioned at a periphery of a display screen

ABSTRACT

An electronic device includes a display screen, which includes a cover glass, a transparent layer disposed under the cover glass disposed under the cover glass, and a display illumination layer disposed under the transparent layer. The electronic device further includes an optical ID sensing module disposed under the display illumination layer and configured to form an image of a fingerprint pattern or a palmprint pattern of a hand of a user. The electronic device further includes a light source disposed at an edge side of the transparent layer, and is configured to emit a light beam to be coupled into the transparent layer through the edge side. A portion of the light beam may be transmitted through the cover glass to illuminate the hand for imaging of fingerprint pattern or the palmprint pattern by the optical ID sensing module.

TECHNICAL FIELD

This patent document generally relates to fingerprint or palmprintrecognition and its applications for secure access of electronic devicesor information systems.

BACKGROUND

Fingerprints can be used to authenticate users for accessing electronicdevices, computer-controlled systems, electronic databases orinformation systems, either used as a stand-alone authentication methodor in combination with one or more other authentication methods such asa password authentication method. For example, electronic devicesincluding portable or mobile computing devices, such as laptops,tablets, smartphones, and gaming systems can employ user authenticationmechanisms to protect personal data and prevent unauthorized access. Inanother example, a computer or a computer-controlled device or systemfor an organization or enterprise should be secured to allow onlyauthorized personnel to access for protecting the information or the useof the device or system for the organization or enterprise. Theinformation stored in portable devices and computer-controlleddatabases, devices or systems, may be personal in nature, such aspersonal contacts or phonebook, personal photos, personal healthinformation or other personal information, or confidential informationfor proprietary use by an organization or enterprise, such as businessfinancial information, employee data, trade secrets and otherproprietary information. If the security of the access to the electronicdevice or system is compromised, these data may be accessed by others,causing loss of privacy of individuals or loss of valuable confidentialinformation. Beyond security of information, securing access tocomputers and computer-controlled devices or systems also allowsafeguard the use of devices or systems that are controlled by computersor computer processors such as computer-controlled automobiles and othersystems such as automatic teller machines (ATMs).

Secured access to a device such as a mobile device or a system such asan electronic database and a computer-controlled system can be achievedin different ways, including, for example, using user passwords.Passwords, however, may be easily stolen or obtained. This nature ofpasswords can reduce the level of the security. Moreover, a user needsto remember a password to use electronic devices or systems, and, if theuser forgets the password, the user needs to undertake certain passwordrecovery procedures to get authenticated or otherwise regain the accessto the device. Such processes may be burdensome to users and may havevarious practical limitations and inconveniences. The personalfingerprint identification can be utilized to achieve the userauthentication for enhancing the data security while mitigating certainundesired effects associated with passwords.

Electronic devices or systems, including portable or mobile computingdevices, may employ user authentication mechanisms to protect personalor other confidential data and prevent unauthorized access. Userauthentication on an electronic device or system may be carried outthrough one or multiple forms of biometric identifiers, which can beused alone or in addition to conventional password authenticationmethods. One form of biometric identifiers is a person's fingerprintpattern or palmprint pattern. A fingerprint sensor and/or a palmprintsensor can be built into an electronic device or an information systemto read a user's fingerprint pattern and/or palmprint pattern, so thatthe device can only be unlocked by an authorized user of the devicethrough fingerprint and/or palmprint authentication.

SUMMARY

According to some embodiments, an electronic device includes a displayscreen. The display screen includes a cover glass, a transparent layerdisposed under the cover glass disposed under the cover glass, and adisplay illumination layer disposed under the transparent layer. Thetransparent layer has an edge side. The electronic device furtherincludes an optical ID sensing module disposed under the displayillumination layer. The optical ID sensing module is configured to forman image of a fingerprint pattern or a palmprint pattern of a hand of auser placed within a field of view (FOV) of the optical ID sensingmodule. The electronic device further includes a light source disposedat the edge side of the transparent layer. The light source isconfigured to emit a light beam to be coupled into the transparent layerthrough the edge side. A portion of the light beam may be transmittedthrough the cover glass to illuminate the hand for imaging offingerprint pattern or the palmprint pattern by the optical ID sensingmodule.

According to some embodiments, an electronic device includes a displayscreen. The display screen includes a cover glass, a touch sensing layerdisposed under the cover glass, and an organic light-emitting diode(OLED) layer disposed under the touch sensing layer. The touch sensinglayer has an edge side. The electronic device further includes anoptical ID sensing module disposed under the OLED layer. The optical IDsensing module is configured to form an image of a fingerprint patternor a palmprint pattern of a hand of a user placed within a field of view(FOV) of the optical ID sensing module. The electronic device furtherincludes a light source disposed at the edge side of the touch sensinglayer. The light source is configured to emit a light beam to be coupledinto the touch sensing layer through the edge side. A portion of thelight beam may be transmitted through the cover glass to illuminate thehand for imaging of fingerprint pattern or the palmprint pattern by theoptical ID sensing module.

According to some embodiments, an electronic device includes a displayscreen. The display screen includes a cover glass, and a displayillumination layer disposed under the cover glass. The electronic devicefurther includes an optical ID sensing module disposed under the displayillumination layer. The optical ID sensing module is configured to forman image of a fingerprint pattern or a palmprint pattern of a hand of auser placed within a field of view (FOV) of the optical ID sensingmodule. The electronic device further includes a light source disposedadjacent the cover glass. The light source is configured to emit a lightbeam to be coupled into the cover glass. A portion of the light beam maybe transmitted through the cover glass to illuminate the hand forimaging of fingerprint pattern or the palmprint pattern by the opticalID sensing module.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a block diagram of an example of an optical sensing basedfingerprint user authentication system that controls the access to acomputer processor controlled device or system.

FIG. 1B is a block diagram showing an exemplary fingerprint sensordevice implementing in a mobile device such as a smartphone based on thedesign in FIG. 1A.

FIG. 2 is a diagram showing an exemplary optical fingerprint sensorpackaged under a screen cover glass of a platform, such as a smartphone.

FIG. 3 is a diagram showing an exemplary fingerprint sensing light path.

FIG. 4 is a diagram of an exemplary optical fingerprint sensor with anair or vacuum coupler.

FIG. 5 is a block diagram showing an exemplary optical fingerprintsensor for fingerprint sensing.

FIG. 6 is a diagram illustrating exemplary live-fingerprint detection.

FIG. 7 shows exemplary extension coefficients of materials beingmonitored.

FIG. 8 shows blood flow in different parts of a tissue.

FIG. 9 shows a comparison between a nonliving material (e.g., a fakefinger) and a live-finger.

FIG. 10 shows a process flow diagram of an exemplary process 1000 forsetting up different security levels for authenticating a live finger.

FIG. 11 is a diagram showing an exemplary optical fingerprint sensor forsensor area decorating.

FIG. 12 is a diagram showing an exemplary optical fingerprint sensorpackaged as a separate button.

FIG. 13 is a diagram showing exemplary fingerprint and live-fingerdetection using the optical fingerprint sensor packaged as a separatebutton.

FIGS. 14 and 15 show examples of devices using LCD and OLED displaymodules in connection with an optical sensor module based on thedisclosed technology.

FIGS. 16, 17, 18, 19, 20 and 21 illustrate examples of features forimplementing an optical sensor module to allow for optical sensing of anobject in contact and non-contact conditions.

FIG. 22 illustrates an example of an optical sensor module to allow foroptical sensing of an object in contact and non-contact conditions inform of a discrete sensor structure similar to the design in FIG. 12.

FIG. 23 illustrates examples of placing the optical sensor module in adevice.

FIG. 24 shows an example of operating an optical sensor module to allowfor optical sensing of an object in contact and non-contact conditions.

FIG. 25 shows two different fingerprint patterns of the same fingerunder different press forces to illustrate the operation of the opticalsensor module for capturing different fingerprint patterns at differenttimes to monitor time-domain evolution of the fingerprint ridge pattern.

FIG. 26 illustrates schematically an electronic platform that includesone or more optical palmprint sensors integrated therein according tosome embodiments.

FIG. 27 illustrates an electronic platform configured to displaysecurity check reminding cursors on a display screen according to someembodiments.

FIG. 28 illustrates an electronic platform configured to displaysecurity check reminding cursors on a display screen according to someembodiments.

FIG. 29 illustrates an electronic platform that includes an opticalpalmprint sensor located near an edge of the frame under a displayscreen according to some embodiments.

FIG. 30 illustrates an electronic platform that includes an opticalpalmprint sensor located under a display screen within a display area ofthe display screen according to some embodiments.

FIG. 31 shows a flowchart illustrating an exemplary method of securitycheck for secure access of an electronic platform using palmprintsensing according to some embodiments.

FIG. 32A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with an optical ID sensing module andperipheral light sources according to some embodiments.

FIG. 32B shows a schematic plan view of the electronic deviceillustrated in FIG. 32A according to some embodiments.

FIG. 33A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with peripheral light sources according tosome embodiments.

FIG. 33B shows a schematic plan view of the electronic deviceillustrated in FIG. 33A according to some embodiments.

FIG. 34A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with an optical ID sensing module andperipheral light sources according to some embodiments.

FIG. 34B shows a schematic plan view of the electronic deviceillustrated in FIG. 34A according to some embodiments.

FIG. 35A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with peripheral light sources according tosome embodiments.

FIG. 35B shows a schematic plan view of the electronic deviceillustrated in FIG. 35A according to some embodiments.

FIG. 36A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with peripheral light sources according tosome embodiments.

FIG. 36B shows a schematic plan view of the electronic deviceillustrated in FIG. 36A according to some embodiments.

FIG. 37A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with peripheral light sources according tosome embodiments.

FIG. 37B shows a schematic plan view of the electronic deviceillustrated in FIG. 37A according to some embodiments.

DETAILED DESCRIPTION

Electronic devices or systems may be equipped with fingerprintauthentication mechanisms to improve the security for accessing thedevices. Such electronic devices or system may include, portable ormobile computing devices, e.g., smartphones, tablet computers,wrist-worn devices and other wearable or portable devices, largerelectronic devices or systems, e.g., personal computers in portableforms or desktop forms, ATMs, various terminals to various electronicsystems, databases, or information systems for commercial orgovernmental uses, motorized transportation systems includingautomobiles, boats, trains, aircraft and others.

Fingerprint sensing is useful in mobile applications and otherapplications that use or require secure access. For example, fingerprintsensing can be used to provide secure access to a mobile device andsecure financial transactions including online purchases. It isdesirable to include robust and reliable fingerprint sensing suitablefor mobile devices and other applications. In mobile, portable orwearable devices, it is desirable for fingerprint sensors to minimize oreliminate the footprint for fingerprint sensing given the limited spaceon those devices, especially considering the demands for a maximumdisplay area on a given device. Many implementations of capacitivefingerprint sensors must be implemented on the top surface of a devicedue to the near-field interaction requirement of capacitive sensing.

Optical sensing modules can be designed to mitigate the above and otherlimitations in the capacitive fingerprint sensors and to achieveadditional technical advantages. For example, in implementing an opticalfingerprint sensing device, the light carrying fingerprint imagininginformation can be directed over distance to an optical detector arrayof optical detectors for detecting the fingerprint without being limitedto the near-field sensing in a capacitive sensor. In particular, lightcarrying fingerprint imagining information can be directed to transmitthrough the top cover glass commonly used in many display screens suchas touch sensing screens and other structures and may be directed forfolded or complex optical paths to reach the optical detector array,thus allowing for flexibility in placing an optical fingerprint sensorin a device that is not available for a capacitive fingerprint sensor.Optical sensor modules based on the disclosed technology in this patentdocument can be an under-screen optical sensor module that is placedbelow a display screen in some designs to capture and detect light froma finger placed on or above the top sensing surface of the screen. Asdisclosed in this patent document, optical sensing can also be used to,in addition to detecting and sensing a fingerprint pattern, detect otherparameters such as whether a detected fingerprint is from a finger of alive person and to provide anti-spoofing mechanism, or certainbiological parameters of the person.

The optical sensing technology and examples of implementations describedin this patent document provide an optical sensor module that uses, atleast in part, the light from a display screen as the illumination probelight to illuminate a fingerprint sensing area on the touch sensingsurface of the display screen to perform one or more sensing operationsbased on optical sensing of such light. A suitable display screen forimplementing the disclosed optical sensor technology can be based onvarious display technologies or configurations, including, a liquidcrystal display (LCD) screen using a backlight to provide while lightillumination to the LCD pixels with optical filters to produce coloredLCD pixels, a display screen having light emitting display pixelswithout using backlight where each individual pixel generates light forforming a display image on the screen such as an organic light emittingdiode (OLED) display screens, or electroluminescent display screens.

Regarding the additional optical sensing functions beyond fingerprintdetection, the optical sensing may be used to measure other parameters.For example, the disclosed optical sensor technology can measure apattern of a palm of a person given the large touch area available overthe entire LCD display screen (in contrast, some designated fingerprintsensors such as the fingerprint senor in the home button of Apple'siPhone/iPad devices have a rather small and designated off-screenfingerprint sensing area that is highly limited in the sensing area sizethat may not be suitable for sensing large patterns). For yet anotherexample, the disclosed optical sensor technology can be used not only touse optical sensing to capture and detect a pattern of a finger or palmthat is associated with a person, but also to use optical sensing orother sensing mechanisms to detect whether the captured or detectedpattern of a fingerprint or palm is from a live person's hand by a “livefinger” detection mechanism, which may be based on, for example, thedifferent optical absorption behaviors of the blood at different opticalwavelengths, the fact that a live person's finger tends to be moving orstretching due to the person's natural movement or motion (eitherintended or unintended) or pulsing when the blood flows through theperson's body in connection with the heartbeat. In one implementation,the optical sensor module can detect a change in the returned light froma finger or palm due to the heartbeat/blood flow change and thus todetect whether there is a live heartbeat in the object presented as afinger or palm. The user authentication can be based on the combinationof the both the optical sensing of the fingerprint/palm pattern and thepositive determination of the presence of a live person to enhance theaccess control. For yet another example, the optical sensor module mayinclude a sensing function for measuring a glucose level or a degree ofoxygen saturation based on optical sensing in the returned light from afinger or palm. As yet another example, as a person touches the LCDdisplay screen, a change in the touching force can be reflected in oneor more ways, including fingerprint pattern deforming, a change in thecontacting area between the finger and the screen surface, fingerprintridge widening, or a blood flow dynamics change. Those and other changescan be measured by optical sensing based on the disclosed optical sensortechnology and can be used to calculate the touch force. This touchforce sensing can be used to add more functions to the optical sensormodule beyond the fingerprint sensing.

With respect to useful operations or control features in connection withthe touch sensing aspect of the display screen, the disclosed opticalsensor technology can provide triggering functions or additionalfunctions based on one or more sensing results from the optical sensormodule to perform certain operations in connection with the touchsensing control over the display screen. For example, the opticalproperty of a finger skin (e.g., the index of refraction) tends to bedifferent from other artificial objects. Based on this, the opticalsensor module may be designed to selectively receive and detect returnedlight that is caused by a finger in touch with the surface of thedisplay screen while returned light caused by other objects would not bedetected by the optical sensor module. This object-selective opticaldetection can be used to provide useful user controls by touch sensing,such as waking up the smartphone or device only by a touch via aperson's finger or palm while touches by other objects would not causethe device to wake up for energy efficient operations and to prolong thebattery use. This operation can be implemented by a control based on theoutput of the optical sensor module to control the waking up circuitryoperation of the display screen which, the pixels are put in a “sleep”mode by being turned off while one or more illumination light sources(e.g., LEDs) for the under-panel optical sensor module or selecteddisplay pixels in an LED display are turned on in a flash mode tointermittently emit flash light to the screen surface for sensing anytouch by a person's finger or palm. Under this design, the opticalsensor module operates the one or more illumination light sources toproduce the “sleep” mode wake-up sensing light flashes so that theoptical sensor module can detect returned light of such wake-up sensinglight caused by the finger touch on the display screen and, upon apositive detection, the entire display screen is turned on or “wokenup”. In some implementations, the wake-up sensing light can be in theinfrared invisible spectral range so a user will not experience anyvisual of a flash light. The display screen operation can be controlledto provide an improved fingerprint sensing by eliminating backgroundlight for optical sensing of the fingerprint. In one implementation, forexample, each display scan frame generates a frame of fingerprintsignals. If, two frames of fingerprint signals with the display aregenerated in one frame when the display screen is turned on and in theother frame when the display screen is turned off, the subtractionbetween those two frames of signals can be used to reduce the ambientbackground light influence. By operating the fingerprint sensing framerate is at one half of the display frame rate in some implementations,the background light noise in fingerprint sensing can be reduced.

In some implementations, an optical sensor module based on the disclosedoptical sensor technology can be coupled to the backside of the displayscreen without requiring creation of a designated area on the surfaceside of the display screen that would occupy a valuable device surfacereal estate in some electronic devices such as a smartphone, a tablet ora wearable device. This aspect of the disclosed technology can be usedto provide certain advantages or benefits in both device designs andproduct integration or manufacturing.

Notably, among other features, the disclosed optical sensing technologycan be implemented to provide optical fingerprint sensing while a userfinger is located near a device while not in contact with the device foruser authentication in accessing the device and can further provideoptical fingerprint sensing while a user finger is in contact with thedevice. In some implementations (e.g., FIGS. 14-16 and 20-21 and theirapplications for optical sensing implementations with LCD and OLEDdisplays), the optical fingerprint sensing can be performed on a fingerin both contact and on-contact instances to enhance the fingerprintsensing and to provide anti-spoofing in the optical sensing. Forexample, multiple fingerprint images can be captured when a finger islocated near a device while not in contact with the device and when thefinger is in contact with the device. The captured fingerprint images ofthe non-contact finger and the captured fingerprint images of thecontact finger provide two different types of optical fingerprintsensing mechanisms and can be used collectively to enhance thefingerprint sensing performance and anti-spoofing feature.

Each user has unique inner topographical features in their fingers thatare below the skin surface and such inner features are not usuallycaptured or available in various fingerprint sensors. Notably, suchunique topographical features below the skin surface are difficult toduplicate by fake fingerprint pattern duplicating techniques many ofwhich are designed to mimic or reproduce external images representingthe external surface pattern of the skin surface such as a 2-dimensionalfingerprint pattern of ridges and valleys on the external surface of afinger. The features of the external surface pattern of ridges andvalleys on the external surface of a finger tend to vary in shape inconnection with the contact conditions of the finger, e.g., a capturedimage of the fingerprint pattern when a finger is not pressed against asurface would tend to reflect the shapes of ridges and valleys of thefinger in their natural positions would be different from the capturedimage of the same finger when a finger is deformed in shape when beingpressed against a surface. Such external fingerprint variation in shapein connection with the contact condition of the finger can vary with theamount or level of pressing when the finger is pressed under differentpressing forces or conditions, thus further complicating the fingerprintdetectability or reliability in fingerprint sensing.

The disclosed optical sensing technology in this patent document can beused to or implemented to capture unique inner topographical featuresbelow the skin surface in user fingers to improve the detection accuracyof the optical fingerprint sensing and thus the security provided byfingerprint authentication.

FIG. 1A is a block diagram of an example of an optical sensing basedfingerprint user authentication system that controls the access to acomputer processor controlled device or system. The system uses anoptical fingerprint sensor with an array of optical detectors to capturean optical image of received light that carries the fingerprint patternfrom a finger that is touched on the optical fingerprint sensor sensingsurface that is illuminated by an illumination light beam. The systemincludes a fingerprint sensor control circuit that receives the outputsfrom the optical detectors in the optical fingerprint sensor, and adigital fingerprint processing processor which may include one or moreprocessors for processing fingerprint patterns and determining whetheran input fingerprint pattern is one for an authorized user. Thefingerprint sensing system may compare a captured fingerprint to astored fingerprint to enable or disable functionality in a device orsystem that is secured by the fingerprint user authentication system.For example, the fingerprint user authentication system at an ATM maydetermine the fingerprint of a customer requesting to access funds.Based on a comparison of the customer's fingerprint to one or morestored fingerprints, the fingerprint user authentication system maycause the ATM system to allow access to funds and may identify thecustomer in order to associate an appropriate account to credit ordeduct the requested funds. A wide range of devices or systems may beused in connection with the disclosed optical fingerprint sensors,including mobile applications, and various wearable or portable devices(e.g., smartphones, tablet computers, wrist-worn devices), largerelectronic devices or systems, e.g., personal computers in portableforms or desktop forms, ATMs, various terminals to various electronicsystems, databases, or information systems for commercial orgovernmental uses, motorized transportation systems includingautomobiles, boats, trains, aircraft and others. FIG. 1B illustrates anexample for a smartphone or a portable device where the fingerprint userauthentication system is a module integrated to the smart phone.

Fingerprint sensing is useful in mobile applications and otherapplications that use secure access. For example, fingerprint sensingcan be used to provide secure access to a mobile device and securefinancial transactions including online purchases. It is desirable toinclude robust and reliable fingerprint sensors features suitable formobile devices. For example, it is desirable for fingerprint sensors inmobile devices to have a small footprint and thin to fit into the highlylimited space in mobile devices; it is also desirable to include aprotective cover to protect such a fingerprint sensor from variouscontaminants.

The optical sensing technology described in this patent document forfingerprint sensing can be implemented to provide high performancefingerprint sensing and can be packaged in compact sizes to fit intomobile and other small device packages. In capacitive fingerprintsensors, the sensing is based on measuring the capacitance between thesensing electrode and a finger surface due to their capacitive coupling.As the protective cover over the capacitive sensor pixels becomesthicker, the electrical field sensed by each capacitive sensor pixeldisperses quickly in space leading to a steep reduction in the spatialresolution of the sensor. In connection with this reduction of thesensing spatial resolution, the sensor signal strength received at eachsensor pixel also reduces significantly with the increase in thicknessof the protective cover. Thus, when the protective cover thicknessexceeds a certain threshold (e.g., 300 μm), it can become more difficultfor such capacitive sensors to provide a desired high spatial resolutionin sensing fingerprint patterns and to reliably resolve a sensedfingerprint pattern with an acceptable fidelity.

The disclosed technology provides optical fingerprint sensor designs inthin optical fingerprint sensor packages for easy integration into amobile device or other compact devices. In some implementations, theoptical fingerprint sensors of the disclosed technology use matchedlight coupling solutions to provide optical fingerprint sensing at lowcost, high performance, and flexible package structures. The disclosedoptical fingerprint sensors may also be configured to providelive-finger detection to improve the fingerprint sensing security.Examples of implementations of the disclosed technology can be used fora wide range of devices and systems including those with a displaystructure. The optical fingerprint sensor based on the disclosedtechnology can be integrated under the same cover of a display such as atouch sensing display device or be packaged in a discrete device that islocated at various locations on the device. In addition, disclosedoptical fingerprint sensor solutions may be used to provide separatefingerprint sensing when a finger is at a non-contact position and an ina contact position and the fingerprint sensing at both contact andnon-contact positons can be combined to enhance the fingerprint sensingand anti-spoofing.

The performance of the optical fingerprint sensors based on thedisclosed technology is not limited by the package cover thickness thatmay hinder capacitive fingerprint sensors. In this regard, an opticalfingerprint sensor based on the disclosed technology can be implementedinto a thin package by using suitable optical imaging captureconfigurations, including configurations that are free of imaging lensesor prisms that tend to render the optical imaging modules bulky.Implementations of optical fingerprint sensors based on the disclosedtechnology can be provide color matching design features to allow thecolors of the optical fingerprint sensing areas to be in certain desiredcolors, e.g., matching colors of the surrounding structures.

In some implementations, the optical fingerprint sensors of thedisclosed technology can be packaged under the platform screen coverglass without modifying the cover thickness and color. The opticalfingerprint sensor can include an optical sensor array, e.g., a photodiode array, or a CMOS sensor array, and the optical sensor array can bedimensioned to a compact size due to the contribution of the compressedlight path structure. Moreover, the design provides flexibility todecorate the sensor area, for example, with color light illumination.

In some implementations, in addition to the optical sensing of afingerprint, optical sensing of a biometric indication is provided toindicate whether an input of the fingerprint pattern is from a liveperson. This additional optical sensing feature can be used to meet theneeds for defeating various ways that may compromise the secured orauthorized access to fingerprint-protected devices or systems. Forexample, a fingerprint sensor may be hacked by malicious individuals whocan obtain the authorized user's fingerprint, and copy the stolenfingerprint pattern on a carrier object that resembles a human finger.Such unauthorized fingerprint patterns may be used on the fingerprintsensor to unlock the targeted device or system. Hence, a fingerprintpattern, although a unique biometric identifier, may not be by itself acompletely reliable or secure identification. The techniques, devicesand systems described in this document supplement the disclosed opticalsensing based fingerprint authentication technology further improve thesecurity level by using an optical sensing technique to determinewhether the input fingerprint is from a live person.

Fingerprint Sensor Circuitry and Live Finger Detection

FIG. 1B is a block diagram showing an exemplary fingerprint sensordevice 23 implementing in a mobile device such as a smartphone, a tabletor a portable computing device 1 with a touch sensing display screen ortouch panel 10 for both touch sensing user inputs and display images andfunctions of the device 1. This is specific implementation example ofthe general optical fingerprint sensing controlled system in FIG. 1A.The touch panel or sensing display screen 10 can be implemented based onvarious touch sensing display designs, including, a display screenhaving light emitting display pixels without using backlight where eachindividual pixel generates light for forming a display image on thescreen such as an organic light emitting diode (OLED) display screens orelectroluminescent display screens or other display screens such asLCD-based touch sensing display screens. The touch sensing display panelincludes a touch sensing and displaying area for both displaying imagesand contents and for receiving contact inputs from a user.

A fingerprint sensor device marker 21 is shown in FIG. 1B to illustratean exemplary position of the fingerprint sensor device 23 with respectto the mobile device 1. The fingerprint sensor device 23 includes asensing unit or circuitry 2 that performs fingerprint scanning,live-fingerprint detection, and sensing area decorative functions. Thesensing unit 2 is communicatively coupled to processing circuitry 5 thathandles signal flows from the sensing unit 2 and to process the signalsassociated with fingerprint scanning and live-fingerprint judgment, etc.

An interface 6 bridges a signal flow between the fingerprint sensordevice 23 and an application platform or a host device 7, which is thesmartphone 1 in this example. Examples of the application platform 7include the smart phone 1, a tablet computer, a laptop computer, awearable device, and other electronic device where a secure access isdesired. For example, the interface 6 can communicate with a centralprocessor (either directly or through other components, such as a bus oran interface) of the smartphone 1 to provide sensor data from thefingerprint sensor device 23 under the fingerprint sensor device marker21 including fingerprint image data and information indicative ofwhether the detected fingerprint making the contact input belongs to alive fingerprint.

In the illustrated example in FIG. 1B, the sensing unit 2 includes afingerprint sensor 3, a live-fingerprint detector 4, and a lightcoupling and illumination unit 8. The fingerprint sensor 3 captures afingerprint pattern and can be implemented using one or more opticaltechniques. The live-fingerprint sensor 4 can include circuitry foranalyzing fingerprint image dynamics. The live finger sensor 4 caninclude circuitry, such as optical sensors, for sensing additionalbiometric markers, such as heartbeat or heart rate from the scannedfingerprint.

The live finger sensor 4 is designed to detect whether a fingerprint isfrom a finger of a live person and this live finger detection orjudgment is based on the fact that a finger of a live person may exhibitcertain motions or physical traits that are typically associated with alive person, e.g., a pulsing signal due to blood flows through theuser's vessels. For example, blood cells manifest different opticalabsorption spectral signatures at visible wavelengths (e.g., a higheroptical absorption) and near IR wavelengths (e.g., a lower opticalabsorption than that is a visible wavelength). Such different opticalabsorption signatures by blood can be optically captured by the liverfinger sensor 4. Other signatures of blood flows may be reflected bypressure variations in blood vessels. In some implementations, the livefinger sensor 4 can include a pressure sensor, an optical sensor, orother sensors that can detect the moving, stretching, or pulsing of alive finger. For example, an optical sensor can include a light source,such as a light emitting diode (LED) or a laser diode (LD) to emit lightand a light detector, such as a photodiode to detect scattered lightscattered from the finger responsive to the emitted light. When thelight propagates through the finger tissues or the blood cells, thelight is partially absorbed and partially scattered. The live fingermovement or the blood flow causes a change in the light absorptioncross-section. The photodiode detects this kind of change and thedetected signal can be used to indicate whether a fingerprint that isbeing presented to the device is from a live person.

The light coupling and illumination unit 8 creates a probe light beam atthe fingerprint sensing surface which generates a reflected probe lightbeam into an optical sensor array (e.g., a photo diode array or CMOSsensor array) of the sensing unit. The fingerprint signals are generatedwhen the probe light beam meets with the finger skin that touches thesensing surface. The fingerprint sensor 3 acquires the fingerprintsignals by detecting the reflection differences of the probing lightbeam at the sensing surface across a fingerprint pattern where locationsof the skin of fingerprint ridges in a finger in contact with thesensing surface creates a lower optical reflection than the opticalreflections at locations of fingerprint valleys in the finger where thefinger skin does not contact the sensing surface. The spatialdistribution the above reflection differences across the touched sensingsurface by the finger is carried by the reflected optical probe lightbeam as an optical image that is detected by the array of opticaldetectors in the fingerprint sensor 3.

The disclosed technology provides for two fingerprint sensor packagingtechniques to implement fingerprint detection and live-finger detection.The first packaging technique is to package the fingerprint sensor underthe screen cover glass of the platform, such as a smartphone. The secondpackaging technique is to package the fingerprint sensor as a separatefingerprint sensing button.

Fingerprint Sensor Packaged Under the Screen Cover Glass

FIG. 2 is a diagram showing an exemplary optical fingerprint sensorpackaged under a screen cover glass of a platform, which can be acommunication or computing device such as a smartphone, a tablet or aportable electronic device. FIG. 3 further illustrates an exemplaryfingerprint sensing light paths of the device in FIG. 2.

In FIG. 2, the exemplary optical fingerprint sensor 23 is packaged undera top transparent layer 50 which may be a screen cover glass, such as anenhanced cover glass of a platform 1. The location of the opticalfingerprint sensor 23 is shown by a fingerprint sensor mark 21 in thetop-down view in the upper right-hand side of the device surface havinga device display 10 (typically, a touch panel assembly) shown in FIG. 2.The illustrated device surface of the smartphone platform 1 includes thetouch panel assembly 10, other sensors 12, such as a camera, andphysical buttons 14 and 16 on one or more sides for performing certainoperations of the device. There are various structures under the coverglass 50, including, e.g., a color material layer 52, display layers 54(e.g., OLED layers or LCD layers) as part of the display screen in thetouch panel assembly 10, and bottom layers 56 of the display screen inthe touch panel assembly 10. A set of touching sensing layers may alsobe placed to overlay the display layers 54 under the top cover glass 50(e.g., between the display layers 54 and the top cover glass 50) toprovide desired touching sensing functions. Therefore, the opticalfingerprint sensor 23 is placed adjacent to and outside of the displaymodule represented by the display layers 54 but both the opticalfingerprint sensor 23 and the display layers 54 are under the commoncontiguous top glass cover 50.

In the example of the optical fingerprint sensor design in FIG. 2, thepackaging design is different from some other fingerprint sensor designsusing a separate fingerprint sensor structure from the display screenwith a physical demarcation between the display screen and thefingerprint sensor (e.g., a button like structure in an opening of thetop glass cover in some mobile phone designs) on the surface of themobile device. Under the illustrated design in FIG. 2 and FIG. 1B, thefingerprint sensor 23 formed in the area underneath fingerprint sensordevice marker 21 for optical fingerprint is located under the top coverglass or layer 50 so that the top surface of the cover glass or layer 50serves as the top surface of the device as a contiguous and uniformglass surface across both the display screen of the touch displayassembly 10 and the optical detector sensor module 23. In the examplesshown in FIGS. 1-6, the optical sensor module is located on one side ofthe transparent substrate 50 as a glass cover that is contiguous withoutany opening at or near the optical sensor module. This design isdifferent various smartphones with a fingerprint sensor and providesunique features and benefits. This design for integrating opticalfingerprint sensing and the touch sensitive display screen under acommon and uniform surface provides benefits, including improved deviceintegration, enhanced device packaging, enhanced device resistance tofailure and wear and tear, and enhanced user experience. In someimplementations of the optical sensing of fingerprints and other sensingoperations, such as the design example in FIG. 12, the optical sensormodule may be packaged in a discrete device configuration in which theoptical sensor module is embodied a distinct structure that has astructural border or demarcation with the display screen or the topcover glass 50, e.g., a button-like fingerprint sensor structure in anopening of the top glass cover in some mobile phone designs to provide acapacitive fingerprint sensor button or areas. The design in FIG. 12 isbased on all optical sensing or a hybrid sensing with both capacitivesensing and optical sensing and thus is different from other button-likefingerprint sensor structures based on capacitive sensing.

The optical fingerprint sensor 23 disposed under the cover glass 50 caninclude an optical coupler 31 that is made of an optical transparentmaterial with a refractive index nc (greater than 1) and is disposedover a matched color material layer 25, and a probe light source 29 thatemits probe light to illuminate a finger placed over the cover glass 50for optical fingerprint sensing by the optical fingerprint sensor 23.The matched coupler 31, the matched color material layer 25, and theprobe light source 29 are disposed over a circuit 27, such as a flexibleprinted circuit (FPC) with desired circuit elements. Also disposed onthe FPC 27 are one or more light sources 33 that produce probe light forliveness detection as further illustrated in the examples associatedwith FIGS. 7-9, optical detectors 34 such as photo diodes for detectingprobe light from the light sources 33 after interacting with the fingerto provide liveness detection, light sources 35 for decoratingillumination, and an optical detector array 37 of optical detectors suchas a photodiode array for capturing the fingerprint pattern orinformation.

As shown in FIGS. 2 and 3, in some implementations, two optional colormaterial layers 25 and 52 can be provided and designed to be colormatched to each other and used to visually conceal or camouflage opticalfingerprint sensor 23 disposed under the cover glass 50. The colormaterial layer 25 is placed underneath the optical fingerprint sensor 23(e.g., on the lower surface of the transparent coupler 31) and the colormaterial layer 52 is placed under the cover glass 50 and above theoptical fingerprint sensor 23 to cover the area that is not covered bythe color material layer 25 so that the two color-matched materiallayers 25 and 52 collectively form a more or less uniform appearancewhen viewed from the above the cover glass 50. In the examples in FIGS.2 and 3, the top color matched material layer 52 has an opening thatdefines an optical sensing area on the fingerprint sensing surface 45 onthe top of the cover glass 50 to allow for the probe light from thelight source 29 to illuminate a finger placed over the cover glass 50for optical fingerprint sensing, and to allow light from the finger tobe collected by the optical fingerprint sensor 23.

FIG. 3 includes FIG. 3A showing an example of the optical fingerprintsensor 23 and FIG. 3B illustrating optical fingerprint sensing based onreflected probe light for capturing a spatial variation in opticalreflection at valleys and ridges on the exterior of a finger.

As shown in FIG. 3A, the light coupler 31 is fixed onto the cover glass50 and an underlying spacer material 39 placed between the light coupler31 and the lower surface of the cover glass 50 to provide two differentlight coupling functions. First, the light coupler 31 couples the probelight from the light source 29 towards the top of the top cover glass 50to illuminate a finger placed over the cover glass 50 for opticalfingerprint sensing, and, second, the light coupler 31 couples the probelight and other light coming from the finger and the cover glass 50 topass through the light coupler 31 along a different optical path as thebeam A′B′ to reach the optical detector array 37 for optical fingerprintsensing. In the specific design shown in FIG. 3A, the coupler 31 is madefrom a solid transparent material with two angled flat facets, one toreceive light from the probe light source 29 and another one tointerface with the optical detector array 37 to direct returned lightfrom the top sensing surface 45 to the optical detector array 37. Theprobe light source 29 is fixed at a proper position so that the probelight beam or a portion of the probe light beam may be projected intothe coupler 31 at desired angles. In implementations, the coupler 31,the spacer material 39, and the cover glass 50 can each be made ofmultiple layers. The optical detector array 37 is fixed at a properposition to receive the reflected probe light beam as part of thereceived beam A′B′ for capturing the optical image of the fingerprintpattern carried by the reflected probe light beam.

Probe light source 29 projects probe light beam AB into coupler 31 whichfurther directs the probe light beam AB through the opening of theoptional color material layer 52 onto the fingerprint sensing surface 45on the top of the cover glass 50 to illuminate the finger in contact.The light beam AB is coupled into cover glass 50 with the help of thespacer material 39 placed underneath the cover glass 50. When nothing isplaced on the top sensing surface 45 of the cover glass 50, a portion orall of the probe light beam power is reflected into the spacer 39, andthis reflected light enters into coupler 31 and forms the reflectedprobe light beam as part of the received beam A′B′ at the opticaldetector array 37. The reflected probe light beam as part of thereceived beam A′B′ is received by the matched optical sensor array 37(e.g., a photo diode array) which converts the optical image carried bythe reflected probe light beam A′B′ into an array of detector signalsfor further processing.

When a finger 43 touches the sensing surface 45 of the cover glass 50,the fingerprint ridges 73 change the local surface reflectance in thecontact area as shown by FIG. 3B. A portion 61 of the probe lightincident on each finger ridge 73 is refracted as light 65 that isscattered in the finger 43, the rest is reflected as light 67 by thefinger ridge 73. The fingerprint valleys are separate from the sensingsurface 45 and generally do not significantly change the local surfacereflection at the sensing surface 45. The incident light 63 that isincident on the fingerprint valleys is reflected as light 69 by thesensing surface 45. The reflected probe light beam which is part of thereceived light beam A′B′ carries the fingerprint signals. Similarly,when something other than a finger skin touches the sensing surface 45of the cover glass 50, the reflected probe light beam as part of thereceived light beam A′B′ carries the touching material information,which is different from a live fingerprint.

In the example of the optical sensor in FIGS. 2 and 3, the materials ofthe coupler 31, spacer 39, and cover glass 50 may be of a proper levelof optical transparency so that the probe light beam can transmit in andthrough the materials to reach the top sensing surface 45 and, oncereturned back from the top sensing surface 45, can transmit to theoptical detector array 37. The propagation directions of the probe lightbeam to and from the top sensing surface 45 are affected by therefractive index nc of the coupler 31, the refractive index ns of thespacer material 39, the refractive index nd of the cover glass 50, andthe refractive index nf of the touching material such as a person'sfinger.

The desired probe light beam angles may be realized by the proper designof the light source 29 and the end surface tilting angle of the coupler31. The divergent angle of the probe light beam is controlled by thestructures of the light source 29 and the shape of the coupler 31's endsurface.

To obtain a clear fingerprint image without an optical lens, theemitting area of the light source 29 may be designed to be small toeffectuate a point light source in some implementations, or the probelight beam may be collimated in other implementations. A small LED lightsource can be installed as the light source 29 and is located far awayfrom the coupler 31 as practical to achieve this in the optical systemshown in FIG. 3.

The optical structures and configurations of the light source 29, thecoupler 31, the spacer material 39, the cover glass 50, and theplacement of the optical detector array 37 in the optical sensor module,including matching proper refractive indexes (nc, ns, nd, nf) of thematerials in the optical fingerprint sensor and initiating the probelight beam incident angles, can be used to cause the probe light beam tobe totally reflected or partially reflected at the sensing surface 45.For example, such an optical sensor can be designed so that the probelight beam is totally reflected when the touch material is water havinga refractive index of about 1.33 at 589 nm, and partially reflected whenthe touch material is finger skin having a refractive index of about1.44 at 589 nm. Such and other designs can cause a variation in theoptical reflection spatial profile at the ridges and valleys of a fingerin contact with the top sensing surface 45 to obtain a spatial patternin the reflected probe light representing the fingerprint pattern on theouter skin of a finger.

In the example in FIG. 3, the probe beam AB size can be H at theincident end facet of the coupler 31 for receiving the probe light. Theprobe beam size may be W at the sensing surface 45 once being redirectedby the coupler 31 upward to illuminate the sensing surface 45. Bymatching the refractive indexes of all of the materials and the shape ofthe coupler 31 and spacer 39, the illuminated dimension W on the sensingsurface 45 may be set to be greater than H. Under this condition, thereflected probe beam in the received probe light beam A′B′ may have abeam size smaller than the probe light beam at the sensing surface 45caused by a compression due to the refraction of the reflected probebeam from the top sensing surface 45, to the coupler 31 and to theoptical detector array 37. The compression ratio is typically decided byrefractive indexes nc and nd. This is an effective method to image alarge area with a small detector array without using an imaging lens. Inaddition, by adjusting the probe light beam divergent angle and thephoto diode array tilting angle, the compression ratio can be furtheradjusted at all dimensions. The reflection from the coupler-spacerinterface and from the spacer-cover interface constitutes optical noiseand can be removed in the processing of the outputs of the opticaldetectors in the optical sensor array 37.

In some implementations, the probe light source 29 may be modulated toallow for an improved optical detection by the optical fingerprintsensor 23, e.g., implementing a lock-in detection based on themodulation frequency for modulating the probe light source 29. Thematched photo diode array 37 can be designed to have a high efficiencyand to work in various optical illumination environments.

Fingerprint Sensing Via Air or Vacuum Coupler

FIG. 4 is a diagram of an exemplary optical fingerprint sensor 23 a withan air or vacuum coupler. The optical fingerprint sensor 23 a of FIG. 4is similar to the optical fingerprint sensor 23 shown in FIGS. 2 and 3in certain aspects. In the optical fingerprint sensor 23 a, a coupler 32made of air or vacuum (with an index of 1) is implemented rather thanthe coupler 31 of FIGS. 2 and 3 with a transparent material with anindex greater than 1. Also, a light path window may be implemented todirect the probe light to the finger 43.

The probe light source 29 and a matched prism 101 are provided under thetop transparent glass 50 and are structured to cooperate to couple theprobe light beam AB generated by the probe light source 29 towards thesensing surface 45 on the top of the top transparent glass 50. The prism101 is placed between the probe light source 29 and the air or vacuumcoupler 32 and is structured to have a first facet to receive andredirect the initially horizontal probe light beam AB by opticalrefraction at a second opposing angled facet to propagate upward throughthe air or vacuum coupler 32 towards the sensing surface 45. Anoptically transmissive spacer material 39 may be placed underneath thetop transparent glass 45 to facilitate the optical sensing operation bythe optical detector array 37 and, in some implementations, includeanti-reflection coatings to reduce undesired optical reflection in theoptical paths in connection with the optical sensing at the opticaldetector array 37. On the other side of the air or vacuum coupler 32 inthe optical path leading to the optical detector array 37, a secondprism 103 with an angled facet is provided to receive returned lightfrom the sensing surface 45 and to direct the received light, includingthe reflected probe light beam A′B′, towards the optical detector array37 through a second facet of the prism 103. The optical detector array37 (e.g., a photo diode array) produces an array of detector outputsignals for optical sensing. Different from FIG. 2 or 3 where theoptical coupler 31 formed of a solid transparent material includes alower surface to hold the color matched material layer 25 below theoptical fingerprint sensor module 23, the color matched color layer 25in the optical fingerprint sensor 23 a in FIG. 4 is formed on (e.g.,painted on) a substrate 105 located on the lower side of the air orvacuum coupler 32 above the FPC 27. This substrate 105 in theillustrated example in FIG. 4 also provides support for the two prisms101 and 103.

In the optical fingerprint sensor 23 a in FIG. 4, the opticalconfiguration of the cover glass 50 for receiving the probe light isconfigured so that the total internal reflection does not happen in thecover glass 50. Due to differences of the optical interfacing conditionsof the cover glass 50 with respect to fingerprint ridge positons andfingerprint valley positions, when a finger 43 touches the sensingsurface 45, the reflectance at the fingerprint ridge positions differsfrom the reflectance at the fingerprint valley positions. Thisdifference varies spatially and represents a 2-dimensional pattern ofridges and valleys of on the external surface of the finger withdifferent fingerprint signals at different locations that are carried bythe reflected probe beam A′B′.

Because the air or vacuum coupler 32 can be implemented at a relativelylow cost and can be easily made of a range of different sizes by placingthe two prisms 103 and 105 at desired spacings from each other, thisdesign can be used to construct optical touch panels with a range ofdifferent display sizes without substantially increasing the costs.

Fingerprint Sensing—a Sample Design

FIG. 5 includes FIGS. 5A, 5B and 5C and shows an exemplary opticalfingerprint sensor 23 b for fingerprint sensing. FIG. 5A shows thesectional view of different layers of the optical fingerprint sensor 23b, FIG. 5B shows a top view of the same optical fingerprint sensor 23 b,and FIG. 5C shows

The specific design of the optical coupler 31 b in the opticalfingerprint sensor 23 b shown in FIG. 5 is a different design from theoptical coupler 31 b for the optical fingerprint sensor 23 of FIGS. 2and 3. Specifically, one surface 111 of the coupler 31 b on the leftside as shown in FIG. 5A has a curved (spherical or aspheric surface)mirror shape for imaging. A probe light source 30 is placed at the focuspoint of the curved mirror surface 111 of the coupler 31 b so that thelight rays reflected by the ciruved mirror surface 111 are parallel raysor the reflected probe beam is a collimated beam that propagates towardsthe top sensing surface 45 for illuminating a finger. In someimplementations, a pinhole can be used on the probe light source 30 tospatially confine the probe light so that a modified light source 30 aonly projects a portion of the light beam to the curved mirror surface111, and the influence of the scattered light is reduced or eliminated.The coupler 31 b is set to be off center with proper distance D when thecurved surface 111 is fabricated. Therefore, the curved mirror surface111 of the coupler 31 b is tilted properly so that the collimated lightbeam from the curved mirror surface 111 is incident into the spacermaterial 39 and the cover glass 50 with desired angles. For example,divergent light beam ASB is collimated and projected to the sensingsurface 45. The reflected probe light beam A′B′ is detected by the photodiode array 37. correspondingly, the central light SC is reflected backto the optical detector array 37 (e.g., a photo diode array) at or neara center C′.

In the example shown in FIG. 5, the light beams are propagated mostly inthe coupler 31 b. The structure can be made compact and robust. In theexample shown in FIG. 5, the material of the coupler 31 b can be of asingle material, or multiple material compounds.

The optical fingerprint sensor of the disclosed technology can beimplemented to provide one or more of the following features. Theoptical fingerprint sensor includes a light source, a coupler, a spacer,a photo diode array, and a cover glass. The spacer may be made toinclude a glass material, an adhesive material, or may be formed by anair gap or vacuum layer. The coupler may be made to include a glassmaterial, an adhesive material, or a layer of air or vacuum. The coverglass for the optical sensor may be configured as part of the displaycover glass in some designs, or may be a separate cover glass in otherdesigns. Each of the coupler, spacer, and cover glass may includemultiple layers in various implementations.

The disclosed technology provides flexibilities in controlling thesignal contrast in the optical sensing at the optical detector array 37by matching the shapes of the materials and refractive indexes of thematerials. By matching the probe light beam incident angle, divergentangle, and the materials of the involved coupler, spacer and cover glassalong the optical path of the illumination probe light, the probe lightbeam may be controlled to be totally reflected or partially reflected atthe sensing surface for different touching materials.

The disclosed optical fingerprint sensor may be configured to operate toeffectuate a water-free effect when interfacing with a finger foroptical fingerprint sensing. For example, a smartphone cover glass invarious smartphones may have a refractive index of about 1.50. Onedesign is to use a low refractive index material (MgF2, CaF2, Polymeretc.) to form the coupler 31 or 31 b in the above design examples. Forexample, the disclosed technology can be used to control the local probelight beam incident angle at the sensing surface 45 of the cover glass50 to be about 68.5□. The total reflection angle is about 62.46□ whenwater is present on or in contact with the sensing surface 45 of theoptical fingerprint sensor, and the total reflection angle is about73.74□ when the ridges of a fingerprint touch the sensing surface 45.The total reflection angle is about 41.81□ when nothing touches thesensing surface 45. In this design, at the water soaking area on the topsensing surface 45, the probe light is totally reflected towards thephoto diode array 37 at locations where the fingerprint ridges touch thetop sensing surface 45 so that less than 5% of the probe light isreflected to the photo diode array 37; and at the dry fingerprintvalleys positions, the probe light beam is also totally reflected to thephoto diode array 37. Under this design, the optical reflection variesfrom the ridges to valleys of the finger and reflection caused by thefingerprint ridges generates stronger optical signals that are detectedto create a high contrast optical image of the fingerprint pattern atthe photo diode array 37.

Human sweat has a refractive index that is lower than the finger's skin.Therefore, based on the differences in optical reflection in the abovedesign, the disclosed technology provides a solution to distinguishingthe sweat pores in the fingerprint. When an air gap is used to form thecoupler such as the example shown in FIG. 4, the total reflection at thesensing surface does not occur. The reflectance difference amongdifferent touching materials (the fingerprint ridges, fingerprintvalleys, and other contaminations) can be used to detect the fingerprintimage.

Due to the light path compression effect in the above optical designs inFIGS. 2 through 5, the sensing area size at the sensing surface 45 onthe cover glass 50 may be greater than the photo diode array size of thephoto diode array 37. The light path compression effect can be utilizedto design the coupler 31 or 31 b to be very thin, thus reducing theoverall thickness of the optical sensing module. For example, less than1 mm thickness CaF2 coupler can be used to realize a 10 mm sensing areasize on the top sensing surface where the image compression ratio can beset around 1:10 by designing the various components in the opticalsensing module. This feature can be used to reduce the sensor thicknessand the sensor cost. In the examples in FIGS. 2 through 5, the photodiode array 37 is installed on one end of the coupler 31 or 31 b insteadof under the coupler. This design leaves the flexibility to apply colorpaint, illumination light etc. to compensate the color or decorate thesensor area.

In implementations, the light source for optical sensing may be a pointlight source installed at a proper distance. In some implementations,the probe light beam may be collimated by spherical lenses, cylinderlenses, or aspheric lenses. In some implementations the light source beplaced a distance to be sufficiently far away from the sensing area 45.The probe light beam may be of a proper divergent angle in some designs.The probe light beam may be divergent or convergent in various designs.

In some implementations, the probe light source may be modulated toimprove the optical sensing by reducing the influence of the backgroundlight which is not modulated and thus can be distinguished from themodulated probe light via a phase sensitive detection similar todetection based on a lock-in amplifier. The photo diode array isdesigned to work well in any illumination environments. Under the aboveoptical design, the cover glass thickness does not limit the opticalfingerprint sensing. The principle can be used to build optical touchpanel.

Live-Fingerprint Detection

FIG. 6 shows an exemplary live-fingerprint detection design in anoptical sensing module. The live-fingerprint detection part of theoptical sensing module can be implemented by one or more designatedlight source 33 and one or more designated optical detectors 34 for livefinger detection in the example of the optical sensing module in FIG. 2that are separate from the light source 29 for providing illuminationfor optical fingerprint sensing and the optical detector array 37 foroptical fingerprint sensing. FIG. 6 shows only the placement of the oneor more designated light source 33 and one or more designated opticaldetectors 34 for live finger detection relative to the optical coupler31 without showing other components of the optical sensing module suchas the light source 29 for providing illumination for opticalfingerprint sensing and the optical detector array 37 for opticalfingerprint sensing.

Alternatively, in other implementations, the live-fingerprint detectioncan be performed by the same the light source 29 and the opticaldetector array 37 for fingerprint sensing without using a separateoptical sensing as shown in FIG. 2. The live fingerprint detection inFIG. 6 can be performed by a finger print sensor, such as one of theoptical fingerprint sensors 23 in FIG. 3, 23 a in FIG. 4, or 23 b inFIG. 5, in a way similar to what is now described below in the specificexample in FIG. 6.

In FIG. 6, the one or more light sources 33 and the receivingphotodetector (PD) array 34 are isolated by a matched optical coupler 31so that the emitting light beams from the one or more light sources 33cannot directly reach the photodetector (PD) 34 for sensing whether afingerprint is from a live finger. The optical coupler 31 directs thelight beams from the light sources 33 to propagate through the lightpath window 41 on the top cover glass 50 (which can be formed by anopening of the color material layer 52 on the bottom of the top coverglass 50) and transmit into the touching material 43, for example, afinger. For a live-fingerprint of a live-person, the blood flow 81 inthe finger exhibits certain optical absorption characteristics atdifferent probe wavelengths and also varies with the heartbeat, thepressing force against the sensor, the breathing or other parameters.Accordingly, the received probe light at the optical detector 34 wouldcarry detectable information associated with optical absorptioncharacteristics at different probe wavelengths, the heartbeat, thepressing force against the sensor, the breathing, micro movement of thefinger, or other parameters and thus can be processed to use suchinformation to determine whether a touched object is from a live person.When the probe light beam 83 from the light sources 33 is coupled by theoptical coupler 31 to enter the material being monitored, the tissues inthe material scatter a portion 85 of the probe light 83 into thereceiving PD array 34. By analyzing the signals received, a sequence ofsignals can be obtained and analyzed for live finger detection.

The fingerprint sensor photo diode array 37 may also be used to detectthe scattered light from the touching materials and thus may also beused for live-fingerprint detection. For example, the micro movement ofthe fingerprint can be used to indicate whether the fingerprint is froma live-finger. A sequence of fingerprint images is used to recover thesignal amplitude and bright spots distribution change with time. A fake,non-live-finger manifests different dynamics from a live-finger.

FIG. 7 shows exemplary optical extinction coefficients of materialsbeing monitored in blood where the optical absorptions are differentbetween the visible spectral range e.g., red light at 660 nm and theinfrared range, e.g., IR light at 940 nm. By using probe light toilluminate a finger at a visible wavelength and an IR wavelength, thedifferences in the optical absorption can be captured determine whetherthe touched object is a finger from a live person.

FIG. 8 shows the blood flow in different parts of a tissue. When aperson' heart beats, the pulse pressure pumps the blood to flow in thearteries, so the extinction ratio of the materials being monitored inthe blood changes with the pulse. The received signal carries the pulsesignals. These properties of the blood can be used to detect whether themonitored material is a live-fingerprint or a fake fingerprint.

FIG. 9 shows a comparison between a nonliving material (e.g., a fakefinger) and a live-finger. Referring to FIG. 6, the light source 33 andthe corresponding designed detector 34 in the optical fingerprint sensorcan also operate as a heartbeat sensor to monitor a living organism. Oneor multiple light wavelengths can be provided from the light source 33.When two or more wavelengths of light are used (e.g., red light around660 nm and IR light at 940 nm), the extinction ratio difference can beused to quickly determine whether the monitored material is a livingorganism, such as live fingerprint. In the example shown in FIG. 8B, twolight sources are used to emit probe light at different wavelengths, oneat a visible wavelength and another an IR wavelength as illustrated inFIG. 7.

When a nonliving material touches the optical fingerprint sensor, thereceived signal reveals strength levels that are correlated to thesurface pattern of the nonliving material and the received signal doesnot contain signal components associated with a finger of a livingperson. However, when a finger of a living person touches the opticalfingerprint sensor, the received signal reveals signal characteristicsassociated with a living person, including different strength levelsbecause the extinction ratios are different for different wavelengths.This method does not take long time to know whether the touchingmaterial is a part of a living person. In FIG. 9, the pulse-shapedsignal reflects multiple touches instead of blood pulse. Similarmultiple touches with a nonliving material does not show the differencecaused by a living finger.

The above optical sensing of different optical absorption behaviors ofthe blood at different optical wavelengths can be performed in a shortperiod for live finger detection and can be faster than opticaldetection of a person's heart beat using the same optical sensor.

In LCD displays, the LCD backlighting illumination light is white lightand thus contains light at both the visible and IR spectral ranges forperforming the above live finger detection at the optical sensor module.The LCD color filters in the LCD display module can be used to allow theoptical sensor module to obtain measurements in FIGS. 7, 8 and 9. Inaddition, the designated light sources for producing the illuminationlight for optical sensing can be operated to emit probe light at theselected visible wavelength and IR wavelength at different times and thereflected probe light at the two different wavelengths is captured bythe optical detector array to determine whether touched object is a livefinger based on the above operations shown in FIGS. 7, 8 and 9. Notably,although the reflected probe light at the selected visible wavelengthand IR wavelength at different times may reflect different opticalabsorption properties of the blood, the fingerprint image is alwayscaptured by both the probe light the selected visible wavelength and theprobe light at the IR wavelength at different times. Therefore, thefingerprint sensing can be made at both the visible wavelength and IRwavelength.

In an implementation where the live-fingerprint detection can beimplemented by a designed optical system such as the light source 33 andoptical detector 34 in the example in FIG. 2 that are separate from thelight source 29 and the optical detector array 37 for fingerprintsensing, the designated light source 33 is operated to emit probe lightat the selected visible wavelength and IR wavelength, e.g., at differenttimes, and the reflected probe light at the two different wavelengths iscaptured by the designated optical detector 34 to determine whethertouched object is a live finger based on the above operations shown inFIGS. 7 and 9.

Alternatively, in an implementation, live-fingerprint detection can beperformed by the same the light source 29 and the optical detector array37 for fingerprint sensing without using a separate optical sensingcomponents designated for live finger detection. Under this design usingthe light source 29 and the optical detector array 37 for bothfingerprint sensing and the live-fingerprint detection, the light source29 is operated to emit probe light at the selected visible wavelengthand IR wavelength at different times and the reflected probe light atthe two different wavelengths is captured by the designated opticaldetector 34 to determine whether touched object is a live finger basedon the above operations shown in FIGS. 7 and 9. Notably, although thereflected probe light at the selected visible wavelength and IRwavelength at different times may reflect different optical absorptionproperties of the blood, the fingerprint image is always captured byboth the probe light the selected visible wavelength and the probe lightat the IR wavelength at different times. Therefore, the fingerprintsensing can be made at both the visible wavelength and IR wavelength.

Security Level Set Up

FIG. 10 shows a process flow diagram of an exemplary process 1000 forsetting up different security levels for authenticating a live fingerbased on the disclosed optical sensing technology for fingerprintsensing. Different security level criterions may be set up based on thetype of action requested. For example, a regular action request isrequired to pass security level 1 check. A request for a financialtransaction for an amount below a threshold, such as under $100 paymentneeds to pass security level 2. A financial transaction for an amountover the threshold may require a higher security level clearance.Different security level action is triggered after different safetylevel evaluation. The safety levels corresponding to different securitylevels can be set up by combining different live-finger signatures. Forexample, single light source signals can be used to set up safety level1 gate, two light source signals can be combined to set up safety level2 gate, and so on.

The method 1000 can begin or be triggered when an action is requested(1002). The requested action is analyzed to determine an appropriatesecurity level (1004). When determined that that security level 1 (thelowest security level) is required (1006), the safety trigger level 1 isrequired to be passed (1014). When the fingerprint analysis passes thesafety trigger level 1, the requested action is performed (1024).However, when the fingerprint analysis fails the safety trigger level 1,the requested action is denied (1022).

Similarly, when determined that that security level 2 is required(1008), the safety trigger level 1 is required to be passed (1016). Whenthe fingerprint analysis passes the safety trigger level 1, therequested action is performed (1024). When the fingerprint analysisfails the safety trigger level 1, the requested action is denied (1022).

When determined that that security level 3 is required (1010), thesafety trigger level 1 is required to be passed (1018). If thefingerprint analysis passes the safety trigger level 1, the requestedaction is performed (1024). If, however, the fingerprint analysis failsthe safety trigger level 1, the requested action is denied (1022).

When determined that that security level N is required (1012), thesafety trigger level 1 is required to be passed (1020). If thefingerprint analysis passes the safety trigger level 1, the requestedaction is performed (1024). If, however, the fingerprint analysis failsthe safety trigger level 1, the requested action is denied (1022).

The optical fingerprint sensor of the disclosed technology can beimplemented to perform live-finger detection with various features. Theoptical fingerprint sensor can detect whether the touching material is alive-finger and can improve the security of the sensor. Specified lightsources and detectors can be used to detect whether the object touchingthe sensing area is a live-finger or a nonliving material. When probelight at a single wavelength is used for illumination, the heartbeatdetection or other live finger characteristics (micromovement of thefinger) can be used to provide a reliable criterion to detect whetherthe object touching the sensing area is a live-finger or a nonlivingmaterial, including the fingerprint of a live-finger. When two or morewavelengths are used, the extinction ratio of the wavelengths arecompared to detect whether the object touching the sensing area is alive-finger or a nonliving material, including the fingerprint of alive-finger. The fingerprint sensor light sources and photo diode arraycan be used to detect whether the object touching the sensing area is alive-finger or a nonliving material, including the fingerprint of alive-finger. The dynamic fingerprint images can be used to detectwhether the object touching the sensing area is a live-finger or anonliving material, including the fingerprint of a live-finger. Multiplesecurity level can be set up for different security requirement tasks.

Sensor Area Decorating

FIG. 11 is a diagram showing an exemplary optical fingerprint sensor forsensor area decorating where an optical fingerprint sensor 23 is placedunder the top cover glass 50 and is located adjacent to and outside thedisplay module. When the optical fingerprint sensor 23 is installedunder the cover glass 50 that is structured to include an optical windowthat transmits light for providing the light path for optical sensing.Specifically, a portion of the cover glass' color coating material 52 isremoved to form this optical window for optical sensing. Because thefingerprint sensor detector is arranged to be at one end of the coupler31, the bottom of the coupler 31 may be painted with color layers 25 sothat the color layers 52 and 25 collectively provide a perception of acontiguous structure to a user. The painted color layers 25 can beselected to match with the platform surface color. For example, to usesame color or pattern under the coupler so that the sensor becomesinvisible. In some implementations, the matched coupler 31 may also bepainted with a desired or different color or pattern to achieve certainor different decorative effects or styles. The matched coupler 31 mayalso be painted with certain patterns or signs, such as homing buttonsign.

The design provides an attractive option to further decorate the sensorarea. For example, one or more designated decorating light sources 35may be provided to provide a designed decorating lighting to the opticalsensing area, e.g., emitting light at different colored lightwavelengths to illuminate the sensor area. This decorating lightingfeature can be useful in dark environments when the bell rings on thesmartphone to indicate where the fingerprint sensing area is located.

The optical fingerprint sensor can be implemented to enable variousdecorative elements including the following: the bottom surface of thecoupler can be painted with same color or pattern layers to match withthe platform surface color; the bottom surface of the coupler can bepainted with different color or pattern layers to show new stylesout-looking; and color light sources 35 can be installed around thecoupler to decorate the sensor area.

Fingerprint Sensor Packaged as a Separate Button

As an alternative implementation, the optical fingerprint sensors 23 inFIG. 3, 23 a in FIGS. 4, and 23 b in FIG. 5 placed under a contiguouscover glass 50 can be packaged as a separate physical fingerprint sensorbutton with a physical demarcation with other parts of the cover glass50.

FIG. 12 is a diagram showing an exemplary optical fingerprint sensorpackaged as a separate button that is located on a front side of amobile device where the device display panel is located. This button canfunction, in addition to housing the optical fingerprint sensor module,as a home button for certain operations of the device, a wake-up buttonfor waking up the device from a power saving mode, or other operation ofthe device.

FIG. 13 is a diagram showing exemplary fingerprint and live-fingerdetection using the optical fingerprint sensor packaged as a separatebutton shown in FIG. 12. The optical fingerprint sensor of FIGS. 12 and13 can be implemented as the optical fingerprint sensors 23 in FIG. 3,23 a in FIGS. 4, and 23 b in FIG. 5 but packaged as a separate button.Thus, the fingerprint sensing and live-finger detecting is also the sameas or similar to those described above. A matched coupler 31 is used toset up the photo diode array 37 position and provide package flexibilityto the visible area. The aforementioned features regarding the differentcomponents of the optical fingerprint sensor in FIGS. 12 and 13 may beimplemented substantially the same as the optical fingerprint sensors 23in FIG. 3, 23 a in FIGS. 4, and 23 b in FIG. 5 including the lightsources. However, to implement the optical fingerprint sensor as aseparate button, the rigidity or the strength of the material for thecover glass 51 may be required at a higher level than the designs inFIGS. 3-5 under the contiguous cover glass 50.

The spacer material 39 and the cover glass 51 add a position shift of Dto the probe light beam AB. When the thickness of the cover glass 51 andthe spacer material 19 is reduced to zero, specifically by excluding thecover glass and spacer, the probe light beam shift D is eliminated. Forexample, a 10 mm sensing size can be realized with less than 1 mmthickness CaF₂. Also, the photo diode array 37 should match with thelight path to realize proper resolution and guarantee the performance inall illumination environments.

The optical fingerprint sensor packaged as a separate button shown inFIGS. 12 and 13 can perform the same fingerprint detection andlive-finger detection as the optical fingerprint sensor of FIGS. 2-11.In addition, the optical fingerprint sensor package as a separate buttoncan be implemented to perform the following features.

The cover glass and related spacer material may be implemented toprovide design flexibility in the thickness according to the needs ofvarious applications. In some implementations, a practical package maybe designed not to use cover glass and spacer material. Another examplefor a practical design is to use a thin layer of cover glass to protectthe coupler where the thin cover glass may be of a high hardness. To usecolored glass or other optical materials to build the cover is alsopractical. When designing a compact button that provide the opticalsensor for optical fingerprint sensing with improved security, variousmechanical parts may be integrated to enhance the rigidity or strengthof the module.

The optical fingerprint sensor designs disclosed in this document can beimplemented in various ways (e.g., under a device cover glass alongsidewith the device display or in a button structure) and are a separatesensing module from the device display screen. Such optical sensordesigns do not interfere with operations, engineering or installation ofthe device display screen and do not interfere functions and featuresthat are associated with or integrated with the display screens such astouch sensing user interface operations and structures. As such, thedisclosed optical sensor technology can be used for devices based onvarious display technologies or configurations, including, a displayscreen having light emitting display pixels without using backlightwhere each individual pixel generates light for forming a display imageon the screen such as an organic light emitting diode (OLED) displayscreens including an active matrix organic light emitting diode (AMOLED)display panel, electroluminescent display screens and other displayswith backlighting such as the ubiquitous liquid crystal display (LCD)screens.

FIGS. 14 and 15 illustrate examples of LCD and OLED display screens fordevices that incorporate optical sensing functions based on thedisclosed technology, including optical fingerprint sensing andadditional optical sensing for determining whether an object in contactis from a live person.

FIG. 14 shows an example of a structure of an LCD display panel thatincludes a LCD display panel structure to display images; a LCDbacklighting light module coupled to the LCD screen to producebacklighting light to the LCD screen for display images; and a toptransparent layer formed over the device screen as an interface forbeing touched by a user for the touch sensing operations and fortransmitting the light from the display structure to display images to auser. The LCD) screen structure can be integrated with a touch sensingstructure that provides touch sensing user interface operations inconnection with operating with the device.

As a specific example, FIG. 14 illustrates a smartphone with a LCD-basedtouch sensing display system 1433. The touch sensing display system 1433is placed under a top cover glass 1431 which serves a user interfacesurface for various user interfacing operations, including, e.g., touchsensing operations by the user, displaying images to the user, and anoptical sensing interface to receive a finger for optical fingerprintsensing and other optical sensing operations. The optical sensor module1490 for optical fingerprint sensing and other optical sensingoperations can be placed in various locations of the device, e.g., atone end of the LCD display module 1433 and under the same top glasscover 1431 as shown. The display system 1423 is a multi-layer liquidcrystal display (LCD) module 1433 that includes LCD display backlightinglight sources 134 (e.g., LED lights) that provide the white backlightingfor the LCD module 1433, a light waveguide layer 1433 c coupled to theLCD display backlighting light sources 1434 to receive and guide thebacklighting light, LCD structure layers 433 a (including, e.g., a layerof liquid crystal (LC) cells, LCD electrodes, transparent conductive ITOlayer, an optical polarizer layer, a color filter layer, and a touchsensing layer), a backlighting diffuser 1433 b placed underneath the LCDstructure layers 1433 a and above the light waveguide layer 1433 c tospatially spread the backlighting light for illuminating the LCD displaypixels in the LCD structure layers 1433 a, and an optical reflector filmlayer 1433 d underneath the light waveguide layer 1433 c to recyclebacklighting light towards the LCD structure layers 433 a for improvedlight use efficiency and the display brightness. The example illustratedin FIG. 14 includes a device electronics/circuit module 1435 for the LCDdisplay and touch sensing operations, one or more other sensors 1425such as an optical sensor for monitoring the light level of thesurroundings, optional side buttons 1427 and 1429 for controls ofcertain smartphone operations.

Among various locations for the optical sensor module 1490 disclosed inthis document, in some implementations, the optical sensor module 1490may be placed next to the display as shown in FIGS. 1B, 2, 11 andalongside with the LCD display module 1433 that is either under thecommon top cover glass 1431 (as shown here in FIG. 14 and also in FIGS.1B, 2 and 11) or in a separate discrete structure (FIG. 12). In suchimplementations, the fingerprint sensing area can include a region abovethe top glass cover 1431 near an edge of but within the LCD displaypanel of the LCD display module 1433 by designing probe light sourcesfor the optical sensor module to capture returned probe light from afinger placed in this region in addition to capturing returned probelight from a finger that is directly on top of the optical sensor moduleoutside the LCD display module 1433. This region can be marked to bevisible to a user for placing a finger for fingerprint sensing. In someimplementations, selected LCD pixels in this region can be operated toturn on to mark this region or the border of this region in the LCDdisplay panel to allow a user to identify the region for placing afinger for fingerprint sensing. In other implementations, one or moreillumination light sources may be added underneath the LCD module toproduce illumination light to illuminate the border or the region on thetop glass cover 1431 to be visible to the user. By providing the one ormore illumination light sources, the region can be optically marked foreasy identification by a user for fingerprint sensing regardless whetherthe LCD display is turned off or turned on. The light from LCD pixelsthat is present in this region within the LCD display can also be usedto add illumination light to a finger in addition to the illumination byprobe light that is produced by and projected from the optical sensormodule. FIG. 14 marks the fingerprint sensing region that includes boththe sensing region within an edge of the display panel area and thesensing region outside the display panel area.

FIG. 15 shows an example of an OLED display screen for a device thatincorporates optical sensing functions based on the disclosedtechnology, including optical fingerprint sensing and additional opticalsensing for determining whether an object in contact is from a liveperson. The OLED display screen is part of the OLED display module 1533that is driven by a driver electronic module or circuit 1535. Similar tothe LCD-based device example in FIG. 14, the optical sensor module 1490is provided in FIG. 15 for optical fingerprint sensing and other opticalsensing operations and can be placed in various locations of the device,e.g., at one end of the OLED display module 1533 and under the same topglass cover 1431 as shown. In some implementations, the optical sensormodule 1490 may be placed next to the display as shown in FIGS. 1B, 2,11 and alongside with the LCD display module 1433 that is either underthe common top cover glass 1431 (as shown here in FIG. 14 and also inFIGS. 1B, 2 and 11) or in a separate discrete structure (FIG. 12). Insuch implementations, the fingerprint sensing region can include boththe sensing region within an edge of the display panel area and thesensing region outside the display panel area as illustrated in FIG. 15.The fingerprint sensing region within the OLED display area can bemarked to be visible to a user for placing a finger for fingerprintsensing. In some implementations, selected OLED pixels in this regioncan be operated to turn on to mark this region or the border of thisregion in the OLED display area to allow a user to identify the regionfor placing a finger for fingerprint sensing. In other implementations,one or more illumination light sources may be added underneath the OLEDmodule to produce illumination light to illuminate the border or theregion on the top glass cover 1431 to be visible to the user. Byproviding the one or more illumination light sources, the region withinthe OLED display area can be optically marked for easy identification bya user for fingerprint sensing regardless whether the OLED display isturned off or turned on. The light from OLED pixels that is present inthis region within the OLED display can also be used to add illuminationlight to a finger in addition to the illumination by probe light that isproduced by and projected from the optical sensor module.

In addition to fingerprint detection by optical sensing, the opticalsensor module based on the disclosed technology in this document canalso be implemented to perform optical sensing for measuring otherparameters. For example, the disclosed optical sensor technology can beused not only to use optical sensing to capture and detect a pattern ofa finger that is associated with a person, but also to use opticalsensing or other sensing mechanisms to detect whether the captured ordetected pattern of a fingerprint is from a live person's hand by a“live finger” detection mechanism.

For example, optical sensing of other user parameters can be based onthe fact that a live person's finger tends to be moving or stretchingdue to the person's natural movement or motion (either intended orunintended), the optical absorption characteristics as disclosed in theexamples in FIGS. 7, 8 and 9, or pulsing when the blood flows throughthe person's body in connection with the heartbeat and blood flow. Asexplained with respect to FIGS. 7, 8 and 9, the ratio obtained atdifferent probe wavelengths can be used to determine whether the touchedobject is from a finger of a living person or a fake fingerprint patternof a man-made material.

For example, the optical sensor module may include a sensing functionfor measuring a glucose level or a degree of oxygen saturation based onoptical sensing in the returned light from a finger or palm. Forexample, as a person touches the display screen, a change in thetouching force can be reflected in one or more ways, includingfingerprint pattern deforming, a change in the contacting area betweenthe finger and the screen surface, fingerprint ridge widening, or ablood flow dynamics change. Such changes can be measured by opticalsensing based on the disclosed optical sensor technology and can be usedto calculate the touch force. This touch force sensing adds morefunctions to the optical sensor module beyond the fingerprint sensing.

For another example, a portion of the light from the display pixels(e.g., OLED or LCD pixels) can enter the finger tissues. This part oflight power is scattered by the finger tissues and a part of thisscattered light may be collected by the optical sensor array in theoptical sensor module. The light intensity of this scattered lightdepends on the finger's skin color, or the blood concentration in thefinger tissue. Such information carried by the scattered light on thefinger is useful for fingerprint sensing and can be detected as part ofthe fingerprint sensing operation. For example, by integrating theintensity of a region of user's finger image, it is possible to observethe blood concentration increase/decrease depends on the phase of theuser's heart-beat. This signature can be used to determine the user'sheart beat rate, to determine if the user's finger is a live finger, orto provide a spoof device with a fabricated fingerprint pattern.

As to obtaining information on the user's skin color by optical sensing,measurements of the optical intensities of returned light from a fingerilluminated probe light at different optical wavelengths of the probelight can be used to obtain the skin color information. The differentoptical wavelengths of the probe light for illuminating the finger canbe achieved in different ways when implementing the disclosed opticalsensing technology. For example, the optical sensor module can includedifferent probe light sources at different optical wavelengths. Foranother example, when implementing the optical sensing in a device withan OLED display panel, the OLED display panel contains different colorpixels, e.g., adjacent red, green and blue pixels within one color OLEDpixel and can be controlled to provide desired colored light toilluminate the finger for the measuring the skin color. Specifically,color of pixels within each color pixel of the OLED display panel can beselected to turn on to illuminate the finger at different colors. Thelight intensities of the scattered light by the finger under theillumination of the probe light at different colors/optical wavelengthscan be recorded at the optical sensor array and this intensityinformation at the different optical wavelengths can be used torepresent the user's skin color and can be used as a user identificationparameter. In this regard, when a user registers a finger forfingerprint authentication operation for a device, the opticalfingerprint sensor measures intensities of the scatter light from fingerat two different colors or wavelengths A and B, as measured intensitiesIa and Ib, respectively. The ratio of Ia/Ib could be recorded and storedas a user authentication data point and is used to compare with a latermeasurement of the ratio of Ia/Ib obtained when user's finger is placedon the sensing area as part of the fingerprint sensing operation to gainaccess to the device. This method can help reject the spoof device whichmay not match user's skin color.

For another example, people have unique topographical or tissue featuresin their fingers that are below the skin surface and such features arenot usually captured or available in various fingerprint sensors. Suchunique topographical or tissue features below the skin surface aredifficult to duplicate by fake fingerprint pattern duplicatingtechniques, and such features tend to vary when a finger is not pressedagainst a surface and when a finger is deformed in shape when beingpressed against a surface. The optical sensing based on the disclosedtechnology in this document can be implemented to use probe light at anoptical wavelength that penetrates into a human skin surface (e.g., atan IR wavelength) to capture optical images containing information onthe tissue structures below the skin surface and such captured imagescan be processed to obtain the information on the tissue structuresbelow the skin surface as part of determination of whether the fingerunder measurement is a finger of an authorized user for the electronicdevice to provide anti-spoof fingerprint sensing. In implementations,the disclosed technology can be implemented to provide opticalfingerprint sensing by capturing images in non-contact and contactconfigurations to provide different user authentication mechanism byusing the same optical sensor module.

The user authentication can be based on the combination of the both theoptical sensing of the fingerprint pattern and the positivedetermination of the presence of a live person to enhance the accesscontrol.

With respect to useful operation or control features in connection withthe touch sensing aspect of a display screen, the disclosed opticalsensor technology can provide triggering functions or additionalfunctions based on one or more sensing results from the optical sensormodule to perform certain operations in connection with the touchsensing control over the display screen. For example, the opticalproperty of a finger skin (e.g., the index of refraction) tends to bedifferent from other artificial objects. Based on this, the opticalsensor module may be designed to selectively receive and detect returnedlight that is caused by a finger in touch with the surface of thedisplay screen while returned light caused by other objects would not bedetected by the optical sensor module. This object-selective opticaldetection can be used to provide useful user controls by touch sensing,such as waking up the smartphone or device only by a touch via aperson's finger or palm while touches by other objects would not causethe device to wake up for energy efficient operations and to prolong thebattery use. This operation can be implemented by a control based on theoutput of the optical sensor module to control the waking up circuitryoperation of the display screen. For example, designed extra lightsources for optical sensing and the designed extra light sources may beprovided and, in operation, the the designed extra light sources may beturned on in a flash mode to intermittently emit flash light to thescreen surface for sensing any touch by a person's finger or palm whilethe display screen can be placed in a sleep mode to save power. In someimplementations, the wake-up sensing light can be in the infraredinvisible spectral range so a user will not experience any visual of aflash light.

FIG. 16 shows an example of an electronic device in form of a mobiledevice having an optical fingerprint sensing module based on thedisclosed technology. The optical sensing features in this example canbe applied to other electronic devices, e.g., tablets and other portabledevices and larger electronic devices with optical fingerprint sensing.The device includes a touch sensing display panel assembly 3010 whichincludes a display module having display layers 3054 and bottom layers3056. An optical sensor module 3023 is located near or adjacent to thedisplay panel assembly 3010 to provide a fingerprint sensor area 3021outside the display panel area and a fingerprint sensing region 3022inside the display panel area as a virtual fingerprint sensor area sincethe optical sensor module is located in the fingerprint sensor area 3021outside the display panel area. The device can also include one or moreother sensors 3012 (e.g., a front camera), control buttons such as sidecontrol buttons 3014 for performing various device operations.

In FIG. 16, the illustrated device includes a display module thatdisplays images and contents and receives user contact inputs. Thedisplay module 3010 includes a display panel with different displaylayers 3054 and bottom layers 3056. A top transparent layer 3056 isformed over the display panel with display layers 3054 to provide atouch interface for receiving a user contact input and to allow viewingof the displayed images and contents of the display panel. Asillustrated, a user can place a finger 3043 over the device forfingerprint sensing in accessing the device. The top transparent layer3056 includes an extended section extending beyond at least one end ofthe display panel. An optical sensor module 3023 is placed underneaththe extended section of the top transparent layer 3056 and adjacent tothe one end of the display panel 3010. As disclosed in this patentdocument, the optical sensor module 3023 includes one or more probelight sources to produce probe light to illuminate the extended sectionof the top transparent layer 3050 and an adjacent area above the toptransparent layer 3050 above the display panel so as to illuminate anobject above or in contact with the top transparent layer 3050 foroptical sensing. The field of view of the illuminated area above thedisplay panel is marked as 3025 in FIG. 16 and the corresponding areashown in the top transparent layer 3050 is marked by the fingerprintsensing region 3022 inside the display panel area. This is alsoillustrated in FIGS. 14 and 15 for LCD and OLED display panels. Thisfeature allows a finger to be optically imaged by the optical sensormodule 3023 as the finger is placed in the field of view of theilluminated area above sensing region 3022 of the display panel withoutbeing in contact with the top transparent layer 3050. The optical sensormodule 3023 can also perform optical sensing operation when the fingeris in contact with the top transparent layer 3050.

The optical sensor module 3023 includes an optical sensor array forcapturing optical images from the returned probe light and/or otherlight returned from the finger 3043. The optical sensor array includesoptical detectors, e.g., CMOS photo detectors or photodiodes, to detectreflected light from the object above or in contact with the toptransparent layer to detect a presence of a received contact inputassociated with both (1) a first signal to provide a first indication ofa fingerprint to generate a first signal indicative of an image of aspatial pattern of whether the object is a finger of an authorized userfingerprint and (2) a second signal indicative of a second differentsignal to provide a separate second indication of whether the object isa finger of an authorized user.

The optical sensor module 3023 may include one or more trigger sensorsfor detecting whether an object is present or approaching. Such atrigger sensor can generate a trigger probe 3027 and detected thereturned trigger probe to determine whether an object is approaching thesensor module, and to detect and evaluate the approaching object at aproper distance from the display cover 3050. The trigger probe can be anoptical signal such as a probe light beam. In other implementations, atrigger sensor can be an acoustic trigger sensor that uses a soundsignal as the probe, or an electric signal such as a capacitance sensor.

In implementations, the device in FIG. 16 can include a supporttransparent layer 3052 formed below the top transparent layer 3050 andis engaged to the top transparent layer 3050 as a unified toptransparent cover. As illustrated, the support transparent layer 3052 inthis example includes an opening that is underneath the extended sectionof the top transparent layer 3050 and is located adjacent to the one endof the display panel. The optical sensor module 3023 is placed insidethe opening of the support transparent layer 3052 underneath theextended section of the top transparent layer 3050. The top transparentlayer 3050 and the support transparent layer 3052 may be glasstransparent substrates or high-strength transparent materials includingcrystalized materials. The use of the support transparent layer 3052 canenhance the overall structure strength and to securely hold the opticalsensor module 3023.

Referring to FIGS. 1A and 1B, the device in FIG. 16 includes an opticalsensor controller coupled to the optical sensor module to controloperations of the one or more probe light sources and the optical sensorarray to trigger capturing of different images of the object includingan image of the object when the object is above the top transparentlayer without contacting the top transparent layer as part of the firstsignal and another image of the object when the object is in contactwith the top transparent layer as part of the second signal. The opticalsensor controller processes the captured images of the object, includingboth the captured image of the object when the object is above the toptransparent layer without contacting the top transparent layer as partof the first signal and the other captured image of the object when theobject is in contact with the top transparent layer as part of thesecond signal, to determine whether the object is a finger of anauthorized user for the electronic device.

Various optical fingerprint sensing operations can be performed by usingthe device in FIG. 16. For example, when an object or finger touches thedisplay cover 3050, the optical sensor module 3023 can use the returnedprobe light to capture the images of the object or finger in the regionsabove the areas 3022 and 3021 before the object or finger touches thetop transparent layer 3050. Once the object or finger touches the toptransparent layer 3050, the touch sensor in the display furtherevaluates the object to avoid spoof.

The probe light sources are integrated in the optical sensor module 3023to illuminate the object to generate returned probe light from theilluminated object back to the optical sensor module 3023 for imaging bythe optical sensor array inside the optical sensor module 3023. In someapplications, at least one probe light source may be designed to emitprobe light at an optical wavelength that penetrates into a human skinsurface, e.g., at one or more optical wavelengths in the infrared (IR)or near IR spectral range. Under this operation, the optical sensorarray captures (1) images formed by the probe light at the opticalwavelength that penetrates into a human skin surface and containingtissue structures below the skin surface, and (2) images representing asurface pattern of the skin surface such as a fingerprint pattern ofridges and valleys of a finger. Accordingly, the optical sensorcontroller processes (1) the images formed by the probe light at theoptical wavelength that penetrates into a human skin surface andcontaining tissue structures below the skin surface, and (2) the imagesrepresenting a surface pattern of the skin surface such as a fingerprintpattern of ridges and valleys of a finger to form a 3-dimensionalprofile for determination of whether the object is a finger of anauthorized user for the electronic device to provide anti-spooffingerprint sensing.

This use of the probe light allows imaging of the inner tissues of thefinger to generate a user-specific signature is difficult to duplicateby a fake finger pattern device and can be used as an anti-spoofmechanism as part of the user authentication process for accessing thedevice. In particular, the above user-specific signature containinginner tissue information under the user finger skin is captured duringthe user registration process for the device by using the optical sensormodule 3023 and is stored for comparison in a user access operation. Afake pattern is unlikely to match such a signature due to the use of theinformation of inner tissues of the finger below the skin surface andthe imaging by the same optical sensor module 3023 for capturing theinformation of inner tissues of the finger below the skin surface. Inaddition, a finger exhibits different surface patterns and inner tissuestructures when the finger is free from shape deformation without beingin contact with the top transparent layer 3050 and when the finger ispressed against the top transparent layer 3050 to undergo somedeformation in shape so that using different stored signatures capturedby the optical sensor module 3023 when the finger is not in contact withthe top transparent layer 3050 and when the finger is pressed againstthe top transparent layer 3050 provide enhanced anti-spoof features. Oneaspect of the disclosed technology in this patent document is to usesuch different surface patterns and inner tissue structures includinginformation captured when a finger is not in contact with the topsensing surface to provide improved fingerprint detection security.

In FIG. 16, in addition to illumination provided by the probe light fromthe optical sensor module 3023, the display light from the displaypixels (e.g., LCD or OLED pixels) may also be used to provide additionalillumination for optical sensing operations. In some implementations,one or more extra illumination light sources 3024 may be providedoutside the optical sensor module 3023 to assist with the illuminationof the object or finger. In the example shown in FIG. 16, the one ormore extra illumination light sources 3024 are shown to be located belowthe display module.

One technical challenge in optical fingerprint sensing is the undesiredbackground light, especially when the device in FIG. 16 is used inoutdoor settings or an environment with strong background lighting. Toaddress this, the optical sensor module 3023 can include an opticalfilter above the optical sensor array to transmit the probe light whileblocking background light from reaching the optical sensor array. Forexample, the optical filter may be structured to reduce infrared lightfrom reaching the optical sensor array, a strong background source fromthe sunlight. Such an optical filter can be a bandpass filter or one ormore filter coatings that are integrated in the detection light path.Each illumination light source can be operated in a flash mode toproduce high illumination brightness in a short period time.

in-display optical fingerprint sensing region 3022 inside the displayscreen and the positon of the optical sensor module located outside thedisplay screen which may be implemented by various designs, includingthe design examples in FIGS. 14, 15 and 16. The in-display opticalfingerprint sensing region 3022 is illuminated to be visible to a userand this illumination can be achieved by using the display pixels orextra light sources. In some designs, the optical sensor module positionmay be aligned to be in the frame edge area of the display.

FIG. 18 shows a color coating feature that can be implemented in theoptical sensor module design in FIG. 16. Specifically, FIG. 18 shows amulti-layered structure of the display cover. For example, the cover mayinclude one top layer 3050 and a support layer 3052, which can beengaged to each other via different ways, including using an adhesive.In some designs, the top layer 3050 can be very thin (e.g., 200 to 400microns or other thickness) and the optical sensor module 3023 may besmall, e.g., a dimension of around a few millimeters. A color coating3029 is formed under the top transparent layer inside the opening of thesupport layer 3052. The color coating 3029 may be patterned to includelight source windows 3033 for transmitting probe light from theillumination light sources and a sensing light path window 3035. In somedesigns, the color coating 3029 may be optically opaque. In otherdesigns, the color coating 3029 may be transparent or partiallytransparent to the probe light from the light sources where the windows3033 may not be needed.

in the optical sensor module in FIGS. 16 and 18, including the opticalsensor array 3063 which may be a photodiode array, probe or illuminationlight sources (LEDs etc.) 3065, and related circuits 3069 integrated ona chip board 3061. Flexible printed circuit (FPC) 3071 is bonded ontothe sensor chip board 3061 via bonding pads 3067. Processing electronics3077 and connector 3079 are mounted on the FPC 3071. The FPC 3071 can bepatterned to include openings for light source windows 3075 anddetection light path widow 3073 formed in the color coating 3029 shownin FIG. 18.

In some implementations, the light sources 3065 may be directly mountedunder the FPC 3071. The optical filter for reducing background light canbe optical filter coatings formed on the surface of the photodiode array3063. Furthermore, in some designs, an enhancement side wall structuremay be included in the module.

FIG. 20 shows examples of various details in the structure and operationof the optical sensor module 3023 in FIGS. 16, 17, 18 and 19. Thesupport layer 3052 under the display cover 3050 can be made a throughhole to hold the optical sensor module 3023. The wall of the hole ispainted with color coating 3029 as the sensor module wall that blocksundesired background or environmental light. An optical imaging or lightcollection module 3089 is provided to capture returned light from anobject or finger for imaging by the optical sensor array 3063. Thisoptical imaging module 3089 may include a pinhole or micro lens that ismounted under the cover top layer 3050 in some imlementations. Thesensing light path window 3035, the pinhole/micro lens 3089 and thedetection light path window 3073 can be aligned so that the opticalsensor array 3063 can receive the image signal light 3087 in the fieldof view that covers the in-display fingerprint sensing region 3022.

In some implementations, the light 3081 from light sources 3065, thelight 3083 from display 3054, the light 3085 from extra light source3024 may be used to illuminate the finger. Multiple light wavelengthsare included for the light sources to realize fingerprint detection andanti-spoof function. For example, live finger spectrum signature can beused to check if the finger is alive. For example, if red or near IRlight is used as light source, the sensor can image deeper tissues underthe skin, such as the dermis. With this signature, the fingerprint canbe imaged with sufficient information regardless of the conditions ofthe finger or the sensing surface, dry, wet, or worn-out fingerprintpatterns with shallow finger ridge-valley features. In this approach,the fingerprint can be imaged when the finger is not pressed on thedisplay. In addition to the 2-D fingerprint patterns, the finger profileinformation included in the database also includes 3D fingerprintinformation that contains inner tissue structures of a finger under theskin. Notably, the image of deeper tissue can be difficult to beduplicated in fake fingerprint and therefor the disclosed opticalfingerprint sensing improves the fingerprint detection accuracy withbuilt-in anti-spoofing feature.

FIG. 21 shows examples of capturing images of a finger in contact andnon-contact conditions in the device design in FIG. 16. As illustrated,the optical sensor controller may be operated to trigger capturing ofdifferent images of the object when (1) the object is above the toptransparent layer without contacting the top transparent layer and isapproaching the top transparent layer (top), (2) the object is incontact with the top transparent layer (middle), and (3) the object ismoving away from the top transparent layer (bottom). Those differentimages can be optically captured and used to further improve theanti-spoof function of the fingerprint sensing.

FIG. 22 further shows an example of an optical sensor module designbased on the discrete “button” structure formed in a peripheral area ofthe top transparent cover as shown in FIG. 12.

The optical sensor module designs based on the disclosed technology canbe implemented in various locations on the front facet, back facet andsides of a device and in various configurations. FIG. 23 illustratessome examples. For example, the optical sensor module may be locatedinside a button of the electronic device. In some designs, the button ofthe electronic device is on a side facet, a back facet or a front sideof the electronic device. The button of the electronic device isoperable to perform another operation different from fingerprintsensing, e.g., a power button for turning on or off power of theelectronic device.

FIG. 24 shows a flowchart illustrating one example of a method foroperating an optical sensor module to authenticate a user for accessingan electronic device. This method includes operating one or more probelight sources of the optical sensor module to produce probe light toilluminate an adjacent area of the electronic device; operating anoptical sensor array of optical detectors of the optical sensor moduleto detect reflected light from an object that is present in theilluminated adjacent area to determine the presence of the object; andoperating the one or more probe light sources and the optical sensorarray to perform a first optical fingerprint sensing operation when thepresence of the object is detected while the object is not in contactwith the electronic device to capture one or more first optical imagesof the object to determine whether the captured one or more firstoptical images of the object contain a first stored fingerprint of afinger of an authorized user previously obtained from the authorizeduser by operating the one or more probe light sources and the opticalsensor array when the finger of the authorized user was not in contactwith the electronic device. Based on the above, the access to theelectronic device is denied when the captured one or more first opticalimages of the object are determined not to contain the first storedfingerprint of the authorized user.

The above processing is represented by the processing operations locatedabove the dashed line in FIG. 24.

Next, when the first optical fingerprint sensing operation determinesthat the captured one or more first optical images of the object in thefirst optical fingerprint sensing operation are determined to containthe fingerprint of an authorized user, the method provides additionaluser authentication as illustrated by processing operations locatedbelow the dashed line in FIG. 24.

Specifically, the method includes operating the one or more probe lightsources and the optical sensor array to perform a second opticalfingerprint sensing operation when the object is in contact with theelectronic device to capture one or more second optical images of theobject to determine whether the captured one or more second opticalimages of the object contain a second stored fingerprint of the fingerof the authorized user previously obtained from the authorized user byoperating the one or more probe light sources and the optical sensorarray when the finger of the authorized user was in contact with theelectronic device. Accordingly, the access to the electronic device isdenied when the captured one or more second optical images of the objectare determined not to contain the second stored fingerprint of theauthorized user. And, the access to the electronic device is grantedwhen the captured one or more second optical images of the object aredetermined to contain the second stored fingerprint of the authorizeduser.

The optical sensors for sensing optical fingerprints disclosed above canbe used to capture high quality images of fingerprints to enablediscrimination of small changes in captured fingerprints that arecaptured at different times. Notably, when a person presses a finger onthe device, the contact with the top touch surface over the displayscreen may subject to changes due to changes in the pressing force.

Referring to FIG. 25, the contact profile area increases with anincrease in the press force, meanwhile the ridge-print expands with theincrease in the press force. Conversely, the contact profile areadecreases with an decrease in the press force, meanwhile the ridge-printcontracts or shrinks with the decrease in the press force. FIG. 25 showstwo different fingerprint patterns of the same finger under differentpress forces: the lightly pressed fingerprint 2301 and the heavilypressed fingerprint 2303. The returned probe light from a selectedintegration zone 2305 of the fingerprint on the touch surface can becaptured by a portion of the optical sensors on the optical sensor arraythat correspond to the selected integration zone 2305 on the touchsurface. The detected signals from those optical sensors are analyzed toextract useful information as further explained below.

When a finger touches the sensor surface, the finger tissues absorb thelight power thus the receiving power integrated over the photo diodearray is reduced. Especially in the case of total inner reflection modethat does not sense the low refractive index materials (water, sweatetc.), the sensor can be used to detect whether a finger touches thesensor or something else touches the sensor accidentally by analyzingthe receiving power change trend. Based on this sensing process, thesensor can decide whether a touch is a real fingerprint touch and thuscan detect whether to wake up the mobile device based on whether thetouch is a real finger press. Because the detection is based onintegration power detection, the light source for optical fingerprintsensing at a power saving mode.

In the detailed fingerprint map, when the press force increases, thefingerprint ridges expand, and more light is absorbed at the touchinterface by the expanded fingerprint ridges. Therefore within arelatively small observing zone 2305, the integrated received lightpower change reflects the changes in the press force. Based on this, thepress force can be detected.

Accordingly, by analyzing the integrated received probe light powerchange within a small zone, it is possible to monitor time-domainevolution of the fingerprint ridge pattern deformation. This informationon the time-domain evolution of the fingerprint ridge patterndeformation can then be used to determine the time-domain evolution ofthe press force on the finger. In applications, the time-domainevolution of the press force by the finger of a person can be used todetermine the dynamics of the user's interaction by the touch of thefinger, including determining whether a person is pressing down on thetouch surface or removing a pressed finger away from the touch surface.Those user interaction dynamics can be used to trigger certainoperations of the mobile device or operations of certain apps on themobile device. For example, the time-domain evolution of the press forceby the finger of a person can be used to determine whether a touch by aperson is an intended touch to operate the mobile device or anunintended touch by accident and, based on such determination, themobile device control system can determine whether or not to wake up themobile device in a sleep mode.

In addition, under different press forces, a finger of a living personin contact with the touch surface can exhibit different characteristicsin the optical extinction ratio obtained at two different probe lightwavelengths as explained with respect FIGS. 7, 8 and 9. Referring backto FIG. 25, the lightly pressed fingerprint 2301 may not significantlyrestrict the flow of the blood into the pressed portion of the fingerand thus produces an optical extinction ratio obtained at two differentprobe light wavelengths that indicates a living person tissue. When theperson presses the finger hard to produce the heavily pressedfingerprint 2303, the blood flow to the pressed finger portion may beseverely reduced and, accordingly, the corresponding optical extinctionratio obtained at two different probe light wavelengths would bedifferent from that of the lightly pressed fingerprint 2301. Therefore,the optical extinction ratios obtained at two different probe lightwavelengths vary under different press forces and different blood flowconditions. Such variation is different from the optical extinctionratios obtained at two different probe light wavelengths from pressingwith different forces of a fake fingerprint pattern of a man-madematerial.

Therefore, the optical extinction ratios obtained at two different probelight wavelengths can also be used to determine whether a touch is by auser's finger or something else. This determination can also be used todetermine whether to wake up the mobile device in a sleep mode.

For yet another example, the disclosed optical sensor technology can beused to monitor the natural motions that a live person's finger tends tobehave due to the person's natural movement or motion (either intendedor unintended) or pulsing when the blood flows through the person's bodyin connection with the heartbeat. The wake-up operation or userauthentication can be based on the combination of the both the opticalsensing of the fingerprint pattern and the positive determination of thepresence of a live person to enhance the access control. For yet anotherexample, the optical sensor module may include a sensing function formeasuring a glucose level or a degree of oxygen saturation based onoptical sensing in the returned light from a finger or palm. As yetanother example, as a person touches the display screen, a change in thetouching force can be reflected in one or more ways, includingfingerprint pattern deforming, a change in the contacting area betweenthe finger and the screen surface, fingerprint ridge widening, or ablood flow dynamics change. Those and other changes can be measured byoptical sensing based on the disclosed optical sensor technology and canbe used to calculate the touch force. This touch force sensing can beused to add more functions to the optical sensor module beyond thefingerprint sensing.

Palmprint Sensing

According to some embodiments, an optical ID sensor may be configured toimage and identify palmprints. Similar to fingerprints, palmprints arealso unique for a person. Therefore, palmprints can also be used as abio-ID for secure access to electronic systems. For example, palmprintsidentification may be used to wake up a smart phone, a tablet computer,or a laptop computer that is in a sleep mode, or to grant access to bankaccounts or authorize electronic payments in an electronic financialsystem. Compared to fingerprint identification, palmprint identificationmay image a larger area of a hand. The relative position of the handwith respect to an optical palmprint sensor may not need to be asaccurate. In addition, palmprints may be obtained at various distancesfrom the optical palmprint sensor, so that “three-dimensional” palmprintID data may be obtained. Therefore, security check using palmprint IDmay afford a better user experience, as well as more robust security.

optical palmprint sensors 4013 a and 4013 b integrated therein accordingto some embodiments. Examples of the electronic platform 4000 mayinclude smart phones, tablet computers, laptop computers, wearabledevices, electronic payment systems, or other electronic devices wheresecure access may be desired. The electronic platform 4000 may have afront side 4001 and a back side 4003. The electronic platform 4000 mayalso have one or more side buttons 4019, such as a power on/off buttonand sound volume control buttons. The electronic platform 4000 may alsoinclude a socket (not shown) for plugging in a headphone, or a Bluetoothinterface for interfacing with a wireless headphone.

As illustrated, an optical palmprint sensor 4013 a may be disposed onthe front side 4001 of the electronic platform 400, and configured todetect and image palmprints of a hand 4005 approaching the front side4001. Alternatively or additionally, an optical palmprint sensor 4013 bmay be disposed on the back side 4003 of the electronic platform 4000,and configured to detect and image palmprints of a hand 4009 approachingthe back side 4003. In some embodiments, an optical plamprint sensor maybe located at a side edge of the frame (not shown), so that palmprintsmay be detected and imaged as a hand approaches the side edge.

In some embodiments, the electronic platform 4000 (e.g., a smart phoneor tablet computer) may include a display screen on the front side 4001.The optical palmprint sensor 4013 a on the front side 4001 may beinstalled under the display screen, located either within the displayarea or at a border of the display area (e.g., similar to the opticalfingerprint sensor illustrated in FIGS. 20 and 21). The opticalpalmprint sensor 4013 b on the back side 4003 may be installed under thebackside structure of the frame.

Each optical palmprint sensor 4013 a or 4013 b may include an opticalassembly 4015 and a photodiode array 4017. In some embodiments, theoptical assembly 4015 may include a lens and/or a pinhole (the opticalassembly 4015 may be referred herein as a lens/pinhole assembly). Theoptical assembly 4015 may be configured to form, at a surface of thephotodiode array 4017, an image of at least a portion of a palm. Thephotodiode array 4017 may be configured to convert optical signals intoelectrical signals, which may be stored in a computer memory and/orprocessed by a processor. An image captured by the optical palmprintsensor 4013 a or 4013 b may include patterns of a palm and/or fingers.

The optical palmprint sensor 4013 a or 4013 b may also include opticalspectral filters. The optical spectral filters may be formed on thesurface of the photodiode array 4017 or surfaces of other opticalcomponents. The optical palmprint sensor 4013 a or 4013 b may alsoinclude electronic circuits coupled to the photodiode array 4017. Theelectronic circuits may be formed on a printed circuit board (PCB). Asan example, the optical palmprint sensor 4013 a or 4013 b may includeoptical and optoelectronic components similar to that illustrated inFIG. 20.

Each optical palmprint sensor 4013 a or 4013 b may have a certainangular field of view (FOV) 4007 or 4011, as illustrated by the dashedlines in FIG. 26. In some embodiments, the optical palmprint sensor 4013a or 4013 b may be configured to detect and image palmprints as a hand4005 or 4009 approaches the front side 4001 or the back side 4003 of theelectronic platform 4000 within its FOV and is at an appropriate objectdistance (e.g., from about 0 mm to about 10 mm, or from about 2 mm toabout 6 mm) of the imaging optics. It may or may not require that anypart of the hand 4005 or 4009 physically touch the optical palmprintsensor 4013 a or 4013 b. Additionally or alternatively, the opticalpalmprint sensor 4013 a or 4013 b may be configured to detect and imageplamprints as a hand 4005 or 4009 is holding the electronic platform4000.

The illumination light for imaging palmprints may include ambient lightfrom the environment, light from a display (in cases in which theoptical palmprint sensor 4013 a or 4013 b is integrated with a displayscreen of the electronic platform 4000). In some embodiments, theelectronic platform 4000 may also include one or more light sourcesdisposed adjacent the optical palmprint sensor 4013 a and/or 4013 b. Thelight sources may provide illumination light on a palm, in addition tothe ambient light and the display light. The light sources may beconfigured to provide infrared light, and/or visible light of selectedwavelengths. For example, the light sources may include lasers or LEDs(e.g., similar to the light sources 3024 and 3065 illustrated in FIG.20). As discussed above, by using multiple light sources with differentwavelengths, the liveness of a palm may be determined.

A security check system of the electronic platform 4000 may detect atrigger event indicating that a person intends to access the electronicplatform 4000. According to various embodiments, the trigger event maybe touching of a physical button (e.g., the power on/off button or avolume control button), or plugging in a headphone or turning on awireless headphone. In response to detecting the trigger event, thesecurity check system may evaluate the palmprints acquired by theoptical palmprint sensors 4013 a and/or 4013 b for authentication. Inthis way, accidental waking up of the electronic platform 4000 without auser's intention may be avoided. Therefore, battery power may be betterpreserved.

In the authentication process, the security check system may compare thepalmprints to palmprint ID data stored in a computer memory to determinewhether the palmprints match the palmprint ID data. The palmprint IDdata may be generated from palmprints of an authorized user acquired bythe optical palmprint sensor 4013 a or 4013 b during a registrationprocess.

In some embodiments, the optical palmprint sensor 4013 a or 4013 b maybe configured to continuously detect whether a palm (or a portion of apalm) is within its field of view (FOV), and acquire palmprints when itdetects that the palm is within its FOV. For example, the opticalpalmprint sensor 4013 a or 4013 b may continuously perform imaging. Thesecurity check system may perform image analysis to determine whether apalm (or a portion of a palm) is being imaged. When it is determinedthat a palm is being imaged, the security check system may cause theoptical palmprint sensor 4013 a or 4013 b to acquire the palmprints(e.g., to capture the palmprints imaged on the photodiode array and savethem in a computer memory). Thus, the security check system may evaluatethe acquired palmprints for authentication as soon as it detects atrigger event without waiting for the optical palmprint sensor 4013 a or4013 b to acquire the palmprints. Therefore, a user may have a betteruser experience by gaining access to the electronic platform relativelyquickly.

In some other embodiments, the optical palmprint sensor 4013 a or 4013 bmay be configured to perform imaging and acquire palmprints only afterthe trigger event has been detected. In this manner, computing resourcesand battery power may be better preserved, perhaps at the expense of alonger latent time in granting access.

In some embodiments, the optical palmprint sensor 4013 a or 4013 b maybe configured to detect whether a palm is within a predetermineddistance from the optical palmprint sensor 4013 a or 4013 b, and acquirepalmprints when it detects that the palm is within the predetermineddistance. The predetermined distance may be determined based on theoptical design of the imaging optics of the optical palmprint sensor4013 a or 4013 b. For example, the imaging optics may be designed toform clear images of an object when the object is within a certain rangeof object distances. For instance, the range of object distances may bebetween 0 mm and about 10 mm, or between about 2 mm and about 6 mm.

In some embodiments, the optical palmprint sensor 4013 a or 4013 b maybe configured to acquire multiple palmprints when the palm is at variousobject distances. For example, palmprints may be acquired when the palmis 2 mm, 3 mm, and 4 mm from the optical palmprint sensor 4013 a or 4013b. Similarly, during the registration process, the optical palmprintsensor 4013 a or 4013 b may acquire multiple palmprints of theauthorized user at various object distances. Thus, the palmprint ID datastored in the computer memory may include three-dimensional (3D)information of the palm of the authorized user. In this way, theauthentication process may be sensitive to the 3D aspect of the objectbeing imaged. Thus, the security check system may have anti-spoofingfunctions. For example, the security check system may be able todistinguish a live 3D palm from a 2D photograph of a plam.

In some embodiments, the electronic platform may display security checkreminding cursors on a display screen. For example, as illustrated inFIG. 27, the electronic platform 4021 displays a security checkreminding cursor 4023 on a front display screen 4022. The security checkreminding cursor 4023 may function as a virtual button. When a finger(or another part of a hand 4023) touches the security check remindingcursor 4023, the security check system may be triggered to evaluate thepalmprints of the hand 4023 acquired by the optical palmprint sensor4027.

In some embodiments, the security check system may require that aparticular finger, for example the index finger, to touch the virtualbutton 4023. For instance, the optical palmprint sensor 4027 may belocated at a lower edge of the front display screen 4022. The virtualbutton 4023 may appear at a location on the display screen 4022 suchthat, when a user uses an index finger of a right hand 4025 to touch thevirtual button 4023, a specific portion of the palm may be within theFOV 4029 of the optical palmprint sensor 4027. If the palmprint ID datawas acquired under similar requirements during a registration process,the palmprint evaluation may be more accurate and robust. In some otherembodiments, multiple virtual buttons may be shown on the display screen4022. The security check system may require that multiple fingers touchthe multiple virtual buttons simultaneously, so that the position of thepalm may be limited to a proper location and orientation.

FIG. 28 illustrates an exemplary embodiment in which security checkreminding cursors (virtual buttons) are used to trigger the evaluationof palmprints. In this example, the electronic platform 4031 may be asmart phone or another type of hand-held devices. The optical pamprintsensor 4030 may be positioned on the back side of the electronicplatform 4031. A user may hold the electronic platform 4031 in a handwith the display screen 4032 facing up and the optical palmprint sensor4030 facing the palm 4039 of the hand. A virtual button 4035 may bedisplayed on the display screen 4032 as a security check reminder. Whenthe user touches the virtual button 4035 with a finger 4033 (e.g., athumb), the security check system may be triggered to evaluate thepalmprints acquired by the optical palmprint sensor 4030. In someembodiments, the security check system may require that a particularfinger (e.g., the thumb) touches the virtual cursor 4035, so that aproper portion of the palm 4039 is within the FOV 4037 of the opticalpalmprint sensor 4030.

FIG. 29 illustrates an exemplary embodiment. In this example, theoptical palmprint sensor 4013 may be located near an edge (e.g., abottom edge) of the frame under the display screen on the front side4001 of the electronic platform 4000. The A security check remindingvirtual button may be displayed on the display screen directly above theoptical palmprint sensor 4013. As a finger 4041 approaches the virtualbutton, the security check system may be triggered to evaluate thepalmprints 4045 acquired by the optical palmprint sensor 4013. In thisexample, the palmprints 4045 may comprise primarily fingerprints as thefinger 4041 may be within the FOV 4043 of the optical palmprint sensor4013. In some embodiments, any part of a palm approaching the virtualbutton (not limited to the finger 4041) may trigger the security checksystem to evaluate the palmprints 4045.

According to various embodiments, the security check system may betriggered to evaluate the palmprints 4045 when the finger 4041 (oranother part of the palm) touches the virtual button, and/or when thefinger 4041 approaches the virtual button and is at a proper distance(e.g., 3 mm or 5 mm) above the display screen, and/or when the finger4041 is lifted from the display screen and is at the proper distance(e.g., 3 mm or 5 mm) above the display screen. The latter two scenariosmay be referred to as remote trigger. The optical palmprint sensor 4013may continuously attempt to image the palmprints 4045, but is triggeredto acquire palmprints 4045 for evaluation only when the security checksystem is triggered.

FIG. 30 illustrates another exemplary embodiment. In this example, theoptical palmprint sensor 4013 may be located under the display screenwithin the display area on the front side 4001 of the electronicplatform 4000. A security check reminding virtual button may bedisplayed on the display screen directly above the optical palmprintsensor 4013. As a finger 4041 (or another part of the palm) approachesthe virtual button, the security check system may be triggered toevaluate the palmprints 4045 acquired by the optical palmprint sensor4013.

According to various embodiments, the security check system may betriggered to evaluate the palmprints 4045 when the finger 4041 touchesthe virtual button, and/or when the finger 4041 approaches the virtualbutton and is at a proper distance (e.g., 3 mm or 5 mm) above thedisplay screen, and/or when the finger 4041 is lifted from the displayscreen and is at the proper distance (e.g., 3 mm or 5 mm) above thedisplay screen. The optical palmprint sensor 4013 may continuouslyattempt to image the palmprints 4045, but is triggered to acquirepalmprints 4045 for evaluation only when the security check system istriggered.

FIG. 31 shows a flowchart illustrating an exemplary method of securitycheck for secure access of an electronic platform using palmprintsensing according to some embodiments. Exemplary electronic platformsmay include smart phones, tablet computers, laptop computers, electronicpayment systems, and the like. The electronic platform may include anoptical palmprint sensor, such as an optical imaging system forcapturing palmprints (or fingerprints) of a person attempting to accessthe electronic platform. The optical palmprint sensor may be positionedunder a display screen (e.g., in a display area or on a border of thedisplay area), or as a discrete button separate from the display screen.

At 4051, the person's palm may approach the electronic platform. Forexample, the person's palm may be grabbing the electronic platform,waving at the electronic platform, or moving toward the electronicplatform.

At 4052, a trigger event may be detected. The trigger event may indicatethat the person is trying to access the electronic platform. The triggerevent may include, for example, when the person touches a physicalbutton (e.g., a power on/off button, or a sound volume control button)or one or more virtual buttons (e.g., security check reminding cursors),and/or when the person's palm is within a proper distance from theoptical palmprint sensor (e.g., 0 mm to 10 mm, or 2 mm to 6 mm from theoptical palmprint sensor), and/or when the person's palm makes aparticular gesture (e.g., waving back and forth).

At 4053, in response to detecting the trigger event, the security checksystem may detect palmprints using the optical palmprint sensor. In someembodiments, the optical palmprint sensor may continuously detect thepresence of a palm within its field of view, and acquire palmprints whenit detects the palm within its field of view. For example, the opticalpalmprint sensor may acquire palmprints when the palm approaches theoptical palmprint sensor and reaches a proper distance from the opticalpalmprint sensor (e.g., 3 mm, 5 mm, or the like), and/or when the palmtouches the optical palmprint sensor, and/or when the palm is movingaway from the optical palmprint sensor and reaches a proper distancefrom the optical palmprint sensor. However, the palmprints may beevaluated by the security check system only when a trigger event hasbeen detected. In this manner, as soon as the trigger event occurs, thesecurity check system may evaluate the palmprints for authentication,and determine whether to grant or deny access in a relatively shortamount of time without waiting for the optical palmprint sensor toacquire palmprints. In this manner, accidentally waking up theelectronic platform without the person's intention may also be avoided.In some other embodiments, the optical palmprint sensor may startacquiring palmprints only when a trigger event has been detected. Inthis manner, computing resources and battery power may be betterpreserved, perhaps at the expense of a longer latent time in grantingaccess.

At 4054, the palmprints acquired by the optical palmprint sensor may becompared to the palmprint ID data stored in a memory to evaluate whetherthe palmprints match with the palmprint ID data. The palmprint ID datamay be generated from palmprints of an authorized user acquired by theoptical palmprint sensor during a registration process.

At 4057, if the evaluation at 4054 results in a “fail,” access may bedenied.

At 4055, if the evaluation at 4054 results in a “pass,” anti-spoofingevaluation may be performed. The anti-spoofing evaluation may include,for example, liveness detection, capacitance measurements, sound echodetection, or specific image analysis (e.g., as described above withreferences to FIGS. 6-9). If the anti-spoofing evaluation at 4055results in a “fail,” access may be denied.

At 4056, if the anti-spoofing evaluation at 4055 results in a “pass,”access may be granted.

It should be appreciated that the specific steps illustrated in FIG. 31provide a particular method of security check for secure access of anelectronic platform according to some embodiments. Other sequences ofsteps may also be performed according to alternative embodiments. Forexample, alternative embodiments of the present invention may performthe steps outlined above in a different order. Moreover, the individualsteps illustrated in FIG. 31 may include multiple sub-steps that may beperformed in various sequences as appropriate to the individual step.Furthermore, additional steps may be added or removed depending on theparticular applications. One of ordinary skill in the art wouldrecognize many variations, modifications, and alternatives.

Peripheral Light Sources

According to some embodiments, an electronic device may include anorganic light-emitting diode (OLED) display screen, also referred to asan active matrix OLED (AMOLED) display screen, and an optical ID sensingmodule may be disposed under the OLED/AMOLED display screen. OLED/AMOLEDdisplay screens may present particular challenges to light sourceconfigurations for providing illumination for the optical ID sensingmodule.

FIG. 32A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with an optical ID sensing module 5023according to some embodiments. FIG. 32B shows a schematic plan view ofthe electronic device according to some embodiments. The electronicdevice may be a smart phone, a tablet computer, a laptop computer, andthe like. The display screen may include a cover glass 5011, and a touchsensing layer 5013 disposed under the cover glass 5011. There may be adark coating 5019 applied to the backside of the cover glass 5011 at theborder of the display screen. The touch sensing layer 5013 has edgesides (referred herein as “edges”) at the periphery of display screen.For example, the touch sensing layer 5013 has two short edges 5014 a andtwo long edges 5014 b along a rectangular border of the display screen.The display screen further includes a display illumination layer 5015disposed under the touch sensing layer 5013. The display illuminationlayer 5015 may include organic light-emitting diodes (OLEDs) accordingto some embodiments. In such cases, the display illumination layer 5015is referred to as an OLED layer 5015 or an active matrix OLED (AMOLED)layer 5015. The display screen may also include other material layers5017 disposed under the OLED/AMOLED layer 5015. For example, the othermaterial layers 5017 may include a protection layer.

As illustrated in FIGS. 32A and 32B, the electronic device is equippedwith the optical ID sensing module 5023 disposed under the OLED/AMOLEDlayer 5015. The other material layers 5017 may include a transparentwindow 5021 directly above the optical ID sensing module 5023, so thatsignal light carrying fingerprint or palmprint information may betransmitted therethrough and be detected by the optical ID sensingmodule 5023. The optical ID sensing module 5023 may be configured toform an image of a fingerprint of a finger 5027 or a palmprint of a palm(e.g., as illustrated in FIG. 26) placed on or adjacent the cover glasswithin a field of view (FOV) of the optical ID sensing module 5023. Theoptical ID sensing module 5023 may include a camera, such as a pinholecamera, a lens camera, a lens-pinhole camera, and the like. The opticalID sensing module 5023 may also include an array of pinhole cameras, anarray of lens cameras, or an array of lens-pinhole cameras. For example,the optical ID sensing module 5023 may be similar to the exampleillustrated in FIG. 20. It should be noted that, although only oneoptical ID sensing module 5023 is illustrated in FIGS. 32A and 32B, theelectronic device may be equipped with multiple optical ID sensingmodules 5023 in some embodiments.

It may be desirable to have an illumination light source to illuminate afinger 5027 or a palm, so that the optical ID sensing module 5023 may beable to capture high quality fingerprints or palmprints. Thus, it may bedesirable to provide a light source positioned adjacent the optical IDsensing module 5023. However, the OLEDs in the OLED/AMOLED layer 5015may be sensitive to high intensity light beams. The OLED/AMOLED layer5015 usually include many thin film transistors (TFTs) for addressingindividual pixels. The TFTs may be damaged if a light beam emitted by alight source is directly incident on them or through them. For example,if a light source is placed below the OLED/AMOLED layer 5015 adjacentthe optical ID sensing module 5023 to provide illumination light for theoptical ID sensing module 5023, a light beam emitted by the light sourcemay be incident on the OLED/AMOLED layer 5015 from below, which maydamage the OLEDs and the TFTs in the OLED/AMOLED layer 5015.

To prevent damages to the OLED/AMOLED layer 5015, one or more lightsources 5025 are provided at an edge of the touch sensing layer 5013above the OLED/AMOLED layer 5015, as illustrated in FIGS. 32A and 32B,according to some embodiments. The light sources 5025 may be micro-sizedlight sources. They may be positioned at any edge of the touch sensinglayer 5013. For example, they may be positioned at a short edge of thetouch sensing layer 5013, as illustrated in FIG. 32B. They may also bepositioned at a long edge of the touch sensing layer 5013, or at both ashort edge and a long edge of the touch sensing layer 5013. The lightsources 5025 may be positioned under the dark coating 5019 at the borderof the display screen, so that they may not be visible.

Referring to FIG. 32A, light emitted by the light sources 5025 may becoupled into the touch sensing layer 5013 from the edge. Some light rays5026 coupled into the touch sensing layer 5013 may be transmitted intothe OLED/AMOLED layer 5015 and be scattered and/or diffracted by themicrostructures (e.g., the TFTs) in the OLED/AMOLED layer 5015. Thescattered light 5029 may be refracted through the top surface of thecover glass 5011 to illuminate the finger 5027 (or a palm). Some lightrays (e.g., the light ray indicated by the dashed arrow) coupled intothe touch sensing layer 5013 may be transmitted into the cover glass5011, which may be subsequently reflected by the top surface of thecover glass 5011 (e.g., when the angle of incidence satisfies thecondition for total internal reflection) toward the OLED/AMOLED layer5015. The reflected light rays may be scattered and/or diffracted by themicrostructures in the OLED/AMOLED layer 5015. The scattered light rays5028 may be transmitted through the cover glass 5011 to illuminate thefinger 5027. In this configuration, light beams emitted by the lightsources 5025 are not directly incident on the OLEDs in the OLED/AMOLEDlayer 5015 from below; instead, they are incident on the OLED/AMOLEDlayer 5015 from the top, and only after being refracted and/or reflectedby the touch sensing layer 5013 and the cover glass 5011. Thus,probabilities of damaging the OLEDs in the OLED/AMOLED layer 5015 may beeliminated or reduced.

According to various embodiments, the light sources 5025 may belight-emitting diodes (LEDs), laser diodes (LDs), vertical-cavitysurface-emitting lasers (VCSELs), and the like. The light sources 5025may be configured to emit light in an ultraviolet wavelength range, avisible wavelength range, a near infrared (NIR) wavelength range, andthe like. In some embodiments, the dark coating 5019 may be made to bepartially transparent for the wavelength range of the light sources5025, so that light emitted by the light sources 5025 may be transmittedthrough the dark coating 5019 and be projected to the finger 5027 orpalm.

In some embodiments, the light sources 5025 are configured to emit lightin a wavelength range that is outside the visible wavelengths. Forexample, the light sources 5025 may be configured to emit light in anNIR wavelength range. The optical ID sensing module 5023 may include afilter configured to transmit only light in the NIR wavelength range,and absorb or reflect visible light. Thus, the optical ID sensing module5023 may image fingerprints or palmprints illuminated only by the lightsources 5025, and the background light in visible wavelengths fromambient light and from light emitted by the OLED/AMOLED layer 5015 maybe suppressed.

Besides preventing potential damages to the OLED/AMOLED layer 5015, theperipheral arrangement of the illumination light sources 5025 mayprovide several other advantageous. For example, the optical ID sensingmodule 5023 may be operated with the OLEDs in the OLED/AMOLED layer 5015(either in the entire display screen, or only those within the field ofview of the optical ID sensing module 5023) are turned off. The OLEDsmay be turned off when the optical ID sensing module 5023 is triggeredto perform security check. Thus, the optical ID sensing module 5023 mayimage fingerprints or palmprints that are illuminated only by the lightsources 5025. As illustrated in FIG. 32A, the light rays from the lightsources 5025 may be incident on the finger 5027 at relatively largeangles of incidence. As such, the fingerprints or palmprints captured bythe optical ID sensing module 5023 may exhibit shades in the valleypositions of the skin on the finger 5027 or palm. Thus, the quality andreliability of fingerprint identification may be improved. This may beespecially advantageous in cases in which the finger 5027 or palm beingauthenticated is dry.

In addition, when the display OLEDs are turned off, the microstructuresof the display, such as the TFTs in the OLED/AMOLED layer 5015 and thesensing circuits in the touch sensing layer 5013 may not be seen by theoptical ID sensing module 5023. This may prevent or reduce artifacts inthe fingerprints or palmprints that may be captured by the optical IDsensing module 5023. Thus, the optical ID sensing module 5023 mayperform authentication more accurately and reliably.

Also, some people may not have clear corneum fingerprint or palmprintdue to shallow finger ridge-valley features, but may have coriumfingerprint or palmprint. According to some embodiments, the lightsources 5025 may be configured to emit light in visible or NIRwavelengths, which may penetrate into the finger/palm tissues. Thus, theoptical ID sensing module 5023 may be able to image corium fingerprintor palmprint. As deeper tissues under the skin may be more difficult toimitate using fake materials, the optical ID sensing module 5023 mayalso have anti-spoofing capabilities.

FIG. 33A shows a partial schematic cross-sectional view of a displayscreen of an electronic device according to some embodiments. Theelectronic device may be equipped with an optical ID sensing module (notshown), similar to the optical ID sensing module 5023 illustrated inFIGS. 32A and 32B. FIG. 33B shows a partial schematic plan view of theelectronic device according to some embodiments. The display screen mayinclude a cover glass 5011, a touch sensing layer 5013 disposed underthe cover glass 5011, and a display illumination layer 5015 disposedunder the touch sensor layer 5013, similar to the display screenillustrated in FIGS. 32A and 32B.

The electronic device includes a dark coating 5019 at the bottom surfaceof the cover glass 5011 at a border of the display screen. Theelectronic device includes one or more light sources 5061 (only onelight source 5061 is shown in FIGS. 33A-33B) positioned under the darkcoating 5019, so that they may be invisible. The touch sensor layer 5013may be thick enough that the one or more light sources 5061 may bepositioned against an edge of the touch sensor layer 5013.

As illustrated in FIG. 33A, light emitted by the one or more lightsources 5061 may be coupled into the touch sensor layer 5013. Some lightrays coupled into the touch sensor layer 5013 may be transmitted intothe cover glass 5011. The transmitted light rays 5063 may be reflectedby the top surface of the cover glass 5011, for example when the angleof incidence of the transmitted light rays 5063 satisfies the conditionfor total internal reflection. The reflected light rays 5068 may bedirected toward the display illumination layer 5015, which maysubsequently be scattered or diffracted by the microstructures in thedisplay illumination layer 5015. The scattered or diffracted light rays5067 may be transmitted through the touch sensor layer 5013 and thecover glass 5011. The transmitted rays 5065 may illuminate the finger orpalm.

Some light rays coupled into the touch sensor layer 5013 may betransmitted through the dark coating 5019 into the cover glass 5011(e.g., the dark coating 5019 may be transparent or partially transparentin the wavelength range of the one or more light sources 5061). Thetransmitted light rays 5064 may be refracted out of the top surface ofthe cover glass 5011, for example when the angle of incidence of thelight rays 5064 does not satisfy the condition for total internalreflection. The transmitted light rays 5066 may illuminate a finger orpalm placed adjacent the cover glass 5011 (e.g., as the finger 5027illustrated in FIG. 32A).

Because the light rays 5065 and 5066 illuminating the finger (or palm)propagate in a direction that is nearly perpendicular to the opticalaxis of the optical ID sensing module (not shown), which issubstantially perpendicular to the surface of the cover glass 5011, thebackground of the fingerprint (or palmprint) image may appear dark. Whena finger (or palm) is touching the top surface of the cover glass 5011,the ridge area of the fingerprint (or palmprint) may result in brightimage lines, while the valley area of the fingerprint (or palmprint) mayresult in dark image lines.

FIG. 34A shows a schematic cross-sectional view of a display screen ofan electronic device equipped with an optical ID sensing module 5023according to some embodiments. FIG. 34B shows a schematic plan view ofthe electronic device according to some embodiments. The display screenmay include a cover glass 5011, a touch sensing layer 5013 disposedunder the cover glass 5011, and a display illumination layer 5015disposed under the touch sensor layer 5013, similar to the displayscreen illustrated in FIGS. 32A and 32B. The display illumination layer5015 may include an OLED/AMOLED layer, an LCD layer, and the like. Theelectronic device includes one or more light sources 5031 positioned atan edge of the cover glass 5011. In some embodiments, the cover glass5011 may include indentations at the edge of the cover glass 5011 forreceiving the one or more light sources 5031.

As illustrated in FIG. 34A, light emitted by the one or more lightsources 5031 may be coupled into the cover glass 5011 through the edge.Some light rays coupled into the cover glass 5011 may be refracted outof the top surface of the cover glass 5011, for example when the angleof incidence of the light rays does not satisfy the condition for totalinternal reflection. The refracted light rays 5033 may illuminate afinger or palm placed on or adjacent the cover glass 5011 (e.g., as thefinger 5027 illustrated in FIG. 32A).

Some light rays coupled into the cover glass 5011 (e.g., the light ray5035) may be refracted out of the bottom surface of the cover glass5011, which may be transmitted through the touch sensor layer 5013 andsubsequently be scattered or diffracted by the microstructures in thedisplay illumination layer 5015. The scattered or diffracted light raysmay be transmitted through the touch sensor layer 5013 and the coverglass 5011. The transmitted light rays 5037 may illuminate the finger orpalm as well.

Some light rays coupled into the cover glass 5011 (e.g., the light ray5036) may be reflected by the top surface of the cover glass 5011 (e.g.,when the angle of incidence of the light ray 5036 satisfies thecondition for total internal reflection). The reflected light rays maybe directed toward the display illumination layer 5015, and may bescattered or diffracted by the microstructures in the displayillumination layer 5015. The scattered or diffracted light rays may betransmitted through the touch sensor layer 5013 and the cover glass5011. The transmitted rays 5038 may also illuminate the finger or palm.

Because the light rays 5033, 5037, and 5038 illuminating the finger (orpalm) propagate in a direction that is nearly perpendicular to theoptical axis of the optical ID sensing module 5023 (which isperpendicular to the surface of the cover glass 5011), the background ofthe fingerprint (or palmprint) image may appear dark. When a finger (orpalm) is touching the top surface of the cover glass 5011, the ridgearea of the fingerprint (or palmprint) may result in bright image lines,while the valley area of the fingerprint (or palmprint) may result indark image lines.

In this configuration, light beams emitted by the light sources 5031 arenot directly incident on the OLEDs in the display illumination layer5015 from the bottom; instead, they are incident on the illumination5015 from the top, and only after being refracted and/or reflected bythe touch sensing layer 5013 and the cover glass 5011. Thus,probabilities of damaging the OLEDs in the display illumination layer5015 may be eliminated or reduced.

FIG. 35A shows a partial schematic cross-sectional view of a displayscreen of an electronic device according to some embodiments. Theelectronic device may be equipped with an optical ID sensing module (notshown), similar to the optical ID sensing module 5023 illustrated inFIGS. 32A and 32B. FIG. 35B shows a partial schematic plan view of theelectronic device according to some embodiments. The display screen mayinclude a cover glass 5011, a touch sensing layer 5013 disposed underthe cover glass 5011, and a display illumination layer 5015 disposedunder the touch sensor layer 5013, similar to the display screenillustrated in FIGS. 32A and 32B.

The electronic device includes one or more light sources 5041 (only onelight source 5041 is shown in FIGS. 35A-35B) positioned under the coverglass 5011 at a border of the display screen. The display screen mayinclude a dark coating 5019 applied to the bottom surface of the coverglass 5011 at the border. The one or more light sources 5041 may bepositioned under the dark coating 5019, so that they may not be visible.

The electronic device also includes a light coupler 5043 positionedunder the cover glass 5011 adjacent the one or more light sources 5041.The light coupler 5043 may be configured to couple light emitted by theone or more light sources 5041 into the cover glass 5011. In someembodiments, the light coupler 5043 may have an index of refraction thatis nearly the same or similar to the index of refraction of the coverglass 5011. Thus, light rays emitted from the light source 5041 may notundergo refraction at the interface between the cover glass 5011 and thelight coupler 5043.

In some embodiments, the dark coating 5019 includes one or more windowareas 5044 adjacent the light sources 5041. The dark coating is removedin the window areas 5044, so as to let light emitted from the one ormore light sources 5041 be transmitted therethrough. In some otherembodiments, the dark coating 5019 may be partially transparent for thewavelength range of the one or more light sources 5041. In someembodiments, the electronic device includes another dark coating 5045under the light coupler 5043 and on a side wall of the light coupler5043. Thus, the border of the display screen may appear dark even in thewindow areas 5044.

As illustrated in FIG. 35A, light emitted by the one or more lightsources 5041 may be coupled into the light coupler 5043, and betransmitted through the top surface of the light coupler 5043 into thecover glass 5011. Some light rays transmitted into the cover glass 5011(e.g., the light rays 5046) may be refracted out of the top surface ofthe cover glass 5011 (e.g., when the angle of incidence of the lightrays 5046 does not satisfy the condition for total internal reflection).The refracted light rays 5048 may illuminate a finger or palm placed onor adjacent the cover glass 5011 (e.g., as the finger 5027 illustratedin FIG. 32A). Some light rays transmitted into the cover glass 5011(e.g., the light ray 5047) may be reflected by the top surface of thecover glass 5011 (e.g., when the angle of incidence of the light rays5047 satisfies the condition for total internal reflection). Thereflected light rays 5042 may be incident on and be scattered ordiffracted by the microstructures in the display illumination layer5015. The scattered or diffracted light rays 5049 may be transmittedthrough the touch sensor layer 5013 and the cover glass 5011. Thetransmitted light rays 5049 may illuminate the finger or palm as well.

Because the light rays 5048 and 5049 illuminating the finger (or palm)propagate in a direction that is nearly perpendicular to the opticalaxis of the optical ID sensing module (not shown), which issubstantially perpendicular to the surface of the cover glass 5011, thebackground of the fingerprint (or palmprint) image may appear dark. Whena finger (or palm) touches the top surface of the cover glass 5011, theridge area of the fingerprint (or palmprint) may result in bright imagelines, while the valley area of the fingerprint (or palmprint) mayresult in dark image lines.

In this configuration, light beams emitted by the light sources 5041 arenot directly incident on the display illumination layer 5015 from thebottom; instead, they are incident on the display illumination layer5015 from the top, and only after being refracted and/or reflected bythe touch sensing layer 5013 and the cover glass 5011. Thus,probabilities of damaging the display illumination layer 5015 may beeliminated or reduced.

FIG. 36A shows a partial schematic cross-sectional view of a displayscreen of an electronic device according to some embodiments. Theelectronic device may be equipped with an optical ID sensing module (notshown), similar to the optical ID sensing module 5023 illustrated inFIGS. 32A and 32B. FIG. 36B shows a partial schematic plan view of theelectronic device according to some embodiments. The display screen mayinclude a cover glass 5011, a touch sensing layer 5013 disposed underthe cover glass 5011, and a display illumination layer 5015 disposedunder the touch sensor layer 5013, similar to the display screenillustrated in FIGS. 32A and 32B.

The electronic device includes a dark coating 5019 at the bottom surfaceof the cover glass 5011 at a border of the display screen. The darkcoating 5019 does not extend to the very edge of the bottom surface ofthe cover glass 5011, or has a window adjacent the edge. The electronicdevice includes one or more light sources 5071 (only one light source5071 is illustrated in FIGS. 36A-36B) positioned under the window of thedark coating 5019 adjacent the edge of the cover glass 5011. Lightemitted by the one or more light sources 5071 may be transmitted throughthe window into the cover glass 5011.

As illustrated in FIG. 36A, the edge of the cover glass 5011 adjacentthe one or more light sources 5071 has a slanted surface 5073. Lightrays transmitted into the cover glass 5011 may be reflected by theslanted surface 5073. The reflected light rays (e.g., the light rays5074 and 5075) may propagate forward toward the opposite edge of thecover glass 5011. In some embodiments, the slanted surface 5073 may forman angle θ with respect to the top surface 5072 of the cover glass 5011.The angle θ may range from about 100 degrees to about 175 degrees,according to some embodiments. The light rays transmitted into the coverglass 5011 may undergo total internal reflection at the slanted surface5073. In some embodiments, a high reflection film may be applied to theslanted surface 5073.

Some reflected light rays (e.g., the light ray 5074) may be refractedout of the top surface of the cover glass 5011 (e.g., when the angle ofincidence of the light ray 5074 does not satisfy the condition for totalinternal reflection). The refracted light ray 5076 may illuminate afinger placed adjacent the top surface of the cover glass 5011. Somereflected light rays (e.g., the light ray 5075) may be reflected by thetop surface of the cover glass 5011 (e.g., when the angle of incidenceof the light ray 5075 satisfies the condition for total internalreflection). The reflected light rays 5073 may be directed toward thedisplay illumination layer 5015, which may be scattered or diffracted bythe microstructures in the display illumination layer 5015. Thescattered or diffracted light rays 5075 may be transmitted through thetouch sensor layer 5013 and the cover glass 5011. The transmitted lightrays 5078 may illuminate the finger or palm.

Because the light rays 5076 and 5078 illuminating the finger (or palm)propagate in a direction that is nearly perpendicular to the opticalaxis of the optical ID sensing module (not shown), which issubstantially perpendicular to the surface of the cover glass 5011, thebackground of the fingerprint (or palmprint) image may appear dark. Whena finger (or palm) is touching the top surface of the cover glass 5011,the ridge area of the fingerprint (or palmprint) may result in brightimage lines, while the valley area of the fingerprint (or palmprint) mayresult in dark image lines.

FIG. 37A shows a partial schematic cross-sectional view of a displayscreen of an electronic device according to some embodiments. Theelectronic device may be equipped with an optical ID sensing module (notshown), similar to the optical ID sensing module 5023 illustrated inFIGS. 32A and 32B. FIG. 37B shows a partial schematic plan view of theelectronic device according to some embodiments. The display screen mayinclude a cover glass 5011, a touch sensing layer 5013 disposed underthe cover glass 5011, and a display illumination layer 5015 disposedunder the touch sensor layer 5013, similar to the display screenillustrated in FIGS. 32A and 32B.

The electronic device includes a partially transparent dark coating 5053at the bottom surface of the cover glass 5011 at the border of thedisplay screen. The partially transparent dark coating 5053 may betransparent or partially transparent to the wavelength range of the oneor more light sources 5051, and block light in the visible wavelengths.The partially transparent dark coating 5053 may have a rough texturesuch that the partially transparent dark coating 5053 may scatter lightincident thereon. The electronic device includes one or more lightsources 5051 (only one light source 5051 is illustrated in FIGS.37A-37B) positioned under the partially transparent dark coating 5053,so that they may be invisible.

As illustrated in FIG. 37A, light emitted by the one or more lightsources 5051 may be coupled into the cover glass 5011 through thepartially transparent dark coating 5053. Some light rays (e.g., thelight rays 5056) may be refracted into the cover glass 5011, which maybe subsequently refracted out of the top surface of the cover glass 5011(e.g., when the angle of incidence of the light rays 5056 does notsatisfy the condition for total internal reflection). The refractedlight rays 5058 may illuminate a finger or palm placed adjacent thecover glass 5011 (e.g., as the finger 5027 illustrated in FIG. 32A).Some light rays (e.g., the light ray 5055) may be scattered into thecover glass 5011 by the partially transparent dark coating 5053, and maybe subsequently reflected by the top surface of the cover glass 5011(e.g., when the angle of incidence of the light rays 5055 satisfies thecondition for total internal reflection). The reflected light rays 5052may be incident on and be scattered or diffracted by the microstructuresin the display illumination layer 5015. The scattered or diffractedlight rays may be transmitted through the touch sensor layer 5013 andthe cover glass 5011. The transmitted light rays 5057 may illuminate thefinger or palm.

Because the light rays 5057 and 5058 illuminating the finger (or palm)propagate in a direction that is nearly perpendicular to the opticalaxis of the optical ID sensing module (not shown), which issubstantially perpendicular to the surface of the cover glass 5011, thebackground of the fingerprint (or palmprint) image may appear dark. Whena finger (or palm) is touching the top surface of the cover glass 5011,the ridge area of the fingerprint (or palmprint) may result in brightimage lines, while the valley area of the fingerprint (or palmprint) mayresult in dark image lines.

In this configuration, light beams emitted by the light sources 5051 arenot directly incident on the display illumination layer 5015 from thebottom; instead, they are incident on the display illumination layer5015 from the top, and only after being refracted and/or reflected bythe touch sensing layer 5013 and the cover glass 5011. Thus,probabilities of damaging the display illumination layer 5015 may beeliminated or reduced.

While this disclosure contains many specifics, these should not beconstrued as limitations on the scope of any invention or of what may beclaimed, but rather as descriptions of features that may be specific toparticular embodiments of particular inventions. Certain features thatare described in this patent document in the context of separateembodiments can also be implemented in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. Moreover, the separation of various system components in theembodiments described in this patent document should not be understoodas requiring such separation in all embodiments.

Only a few implementations and examples are described and otherimplementations, enhancements and variations can be made based on whatis described and illustrated in this patent document.

A recitation of “a”, “an” or “the” is intended to mean “one or more”unless specifically indicated to the contrary.

Ranges may be expressed herein as from “about” one specified value,and/or to “about” another specified value. The term “about” is usedherein to mean approximately, in the region of, roughly, or around. Whenthe term “about” is used in conjunction with a numerical range, itmodifies that range by extending the boundaries above and below thenumerical values set forth. In general, the term “about” is used hereinto modify a numerical value above and below the stated value by avariance of 10%. When such a range is expressed, another embodimentincludes from the one specific value and/or to the other specifiedvalue. Similarly, when values are expressed as approximations, by use ofthe antecedent “about,” it will be understood that the specified valueforms another embodiment. It will be further understood that theendpoints of each of the ranges are included with the range.

All patents, patent applications, publications, and descriptionsmentioned here are incorporated by reference in their entirety for allpurposes. None is admitted to be prior art.

1.-15. (canceled)
 16. An electronic device comprising: a display screencomprising: a cover glass having a front surface and a back surfaceopposite to the front surface; and a display illumination layer disposedunder the cover glass; an optical identification (ID) sensing moduledisposed under the display illumination layer, the optical ID sensingmodule configured to form an image of a fingerprint pattern or apalmprint pattern of a hand of a user placed within a field of view(FOV) of the optical ID sensing module; and a light source disposedunder the back surface of the cover glass, the light source configuredto emit a light beam from an emitting surface that is substantiallyperpendicular to the back surface of the cover glass, a portion of thelight beam being coupled into the cover glass through the back surfaceof the cover glass and being transmitted through the front surface ofthe cover glass to illuminate the hand for imaging of the fingerprintpattern or the palmprint pattern by the optical ID sensing module. 17.The electronic device of claim 16, wherein the light beam emitted by thelight source is incident on the back surface of the cover glass at anon-normal incidence angle.
 18. The electronic device of claim 16,further comprising: a light coupler disposed under the cover glass at aborder region of the display screen; wherein the light source isdisposed under the cover glass and adjacent the light coupler, and thelight beam emitted by the light source is coupled into the cover glassthrough the light coupler.
 19. The electronic device of claim 18,further comprising a first dark coating applied to a bottom surface ofcover glass at the border region of the display screen, the first darkcoating exposing the light coupler, wherein the light source is disposedunder the first dark coating.
 20. The electronic device of claim 19,further comprising a second dark coating applied to a bottom surface ofthe light coupler.
 21. The electronic device of claim 16, wherein thecover glass has a slanted surface at an edge thereof, and wherein thelight source is disposed under the cover glass adjacent the edge, theslanted surface configured to reflect the light beam emitted by thelight source and coupled into the cover glass.
 22. The electronic deviceof claim 21, further comprising a high-reflection coating applied to theslanted surface of the cover glass.
 23. The electronic device of claim21, wherein the slanted surface forms an angle with respect to a topsurface of the cover glass, the angle ranging from about 100 degrees toabout 175 degrees.
 24. The electronic device of claim 21, furthercomprising a dark coating applied to a bottom surface of the cover glassadjacent the edge, the dark coating exposing the light source.
 25. Theelectronic device of claim 16, wherein the light source is configured toemit the light beam in a near infrared (NIR) wavelength range, anultraviolet (UV) wavelength range, or a visible wavelength range. 26.The electronic device of claim 16, wherein the display illuminationlayer comprises a plurality of organic light-emitting diodes (OLEDs).27. The electronic device of claim 16, wherein the display screenfurther comprises a touch sensing layer disposed between the cover glassand the display illumination layer.